Testing and assessing expert software developers and engineers

Programmers ordinarily have a conversation around the project with the customer. They may reject opportunities that require them to be assessed by tests with an eye roll, facepalm, even get offended. ChatGPT made online tests obsolete. What now?

Applied for a job, received a test to complete, told them to get f*cked

—anon. software engineer (ehm, me)

This article comes with a list of examples of tech tests going wrong, a compiled top 10 list of legitimate tools and tips for unobtrusively assessing software engineers as well as extensive footnotes for non-tech people, customers and recruiters alike.

If I go to and old doctor now, I tell him and show him my problem and keep quiet. With respect and even if I genuinely studied my ailment I don’t diagnose myself proposing a cure or ask him to name different knee ligaments.

As a software and web developer once I establish a relationship with the client I figure out how can I help them and propose what needs doing and at what price and do it. I used to write a SWOT ^[1] or do an initial kanban ^[2] board, but a list of tasks suffices. Good client lets me get on with the agreed work after an initial conversation.

At the start of a project often customer has a wrong idea of what needs doing. His developers deeply involved with the project are even deeper in the rabbit hole doing the wrong thing. They often get to a point where they can’t achieve what they wanted with their tool of choice, then seek to hire a person looking for particular tool knowledge ignoring the fact that they may be using the wrong tool and most of the time fixing the wrong problem. The buy-in and investment keeps them going the wrong direction.

With that out of the way my list of examples of testing going wrong

• I once got hit by car and fell off my cycle en route to get to a test interview, run with the bike on my shoulder to catch a bus and was refused by bus driver with it, stopped a builders lorry and dropped the bike on a pile of sand at the back, caught my train to London, run to Old Street, got there on time looking like nothing happened, didn't get the job

• When I was a lad, 18yr old student with a first baby to feed, a well known Norwegian web browser maker sent me an assessment making CSS templates that took my entire weekend to complete instead of focusing on client work. Another notch for packing bags on wife’s list of my failures.

• Junior devs of a UK subsidiary of a large Russian gas company were testing me for Pandas ^[3] knowledge, wanting to push their meter readings analysis prototype to production system when they needed a solid relational time series database and solid systems engineering using strong data types in place based on their research prototypes.

• Developers at a government department were wanting Terraform ^[4] knowledge, where they needed to get their AWS ^[5] infrastructure cost under control best by moving away from it altogether and do it the hard way establishing use of each VM ^[6] instance and not abusing micro-services [7]. Not to mention they put me head to head with a psychopath candidate who spiked my water while I went for a pee during an hour long test.

• A large online product search engine wanted XML sitemaps ^[8] specifically seeking this knowledge when their website had poor information architecture and lots of temporary redirects preventing Google from crawling and assigning link value properly.

• A Belgian organisation looking for an internal coach spent a lot of time during an interview talking about Pydantic ^[9] when their whole organisation was really diversified across Python use and needed a Django ^[10] crash course and internal guidelines to connect up their Python apps.

Advice for assessing software engineers

Assessing a software engineer really is similar to evaluating a doctor, where you may read their published peer reviewed papers and online reviews. Prudence is important and with that, due diligence.

1. Google them. Check their website. Is it solid and well presented?

2. Look at their reviews. What do their customers say? There may be some on their LinkedIn.com profile too.

3. Look up and call up their past customers and employers, people they managed. Your diligence can only be appreciated and you may make an industry connection.

4. Look at their Github.com profile, a website that would host their open source programming contributions. You can look them up by their handle, usually first part of their email address. There will be forks and stars on their code repositories if they are any good.

5. Look at their coder’s rank profile, yes there’s a website ranking developers, codersrank.io

6. See their contributions at stackoverflow.com. They would have answered questions of newbie developers and their answers would have been voted for by the community of software engineers. They would have asked good questions themselves.

7. Look up their email in your favourite search engine and see how they advised on mailing lists in open source community where they would have collaborated on development of software tools used in their daily job to improve them. Keep in mind they may have multiple email addresses and some not in use anymore. Ageing content dropping out of search indexes makes this digging harder plus we reached the stalking boundary.

8. Now if you do test them, pay them in cash their hourly rate for a consultation plus any travel cost. Set aside say 100 EUR per candidate for hour long interviews and just pay saying you appreciate their time whether you take them or not. Best test by using your real past problems to see if they can give you a batter solution or advice. Candidates looking for a job often are in a dire financial situation, so don’t abuse your position. This also massively increases your chances of them accepting shall you make an offer because you bought yourself a bit of time.

9. Cover their travel and food costs, lodging if needed and make sure to take care of their physical safety and well being. See the spiked drink example.

I hope this will help you assessing a programmer you are about to hire and wish you best of luck in this endeavour.

[1] SWOT is a business case document listing strengths, weaknesses, opportunities and threats.

[2] Kanban is a “board” in Japanese a real or virtual one usually with short task oriented tickets in columns named “to do”, “in progress” and “done” where tickets move left to right as project progresses.

[3] Pandas is a software package used by data researchers to glean knowledge from data. It is meant for research and is known to be unstable handling data types of columns when wrong data comes in much like Excel and any other tool guessing datatype of a column.

[4] Terraform is a tool used to declaratively manage virtual cloud computer infrastructure with cumbersome and hard to read repetitive configuration files that describes its final state.

[5] Amazon Web Services, Jeff Bezos controlled platform providing virtual computers and their services holding the industry by the balls because IT people were too lazy managing real iron forged computer servers and mostly forgot how it feels or what a chicken is meant to taste like.

[6] Virtual machine, overpriced time-share piece running on a larger computer a developer never seen

[7] Micro services, theory goes that a large system is meant to be made of smaller independent systems called microservices that don’t have much in common, just talk in agreed formats, so that these can grow independently. A microservice usually has its own VM or a container. Megalomaniac goblin minds tend to setup a bunch of these ahead of time before they are needed where they were meant to design a simple integrated system. This makes development of said systems next to impossible.

[8] XML sitemap is a list of all links or URLs on a website optionally along with their priority value and last update date, it helps search engine crawlers to scan the website and usually sits hidden at an address such as unilexicon.com/fm/sitemap.xml. XML stands for extensible markup language, developed by Microsoft, it uses <tags> and their attributes and helps systems talk together. It looks like this <url><loc>https://unilexicon.com/fm/</loc>…

[9] Pydantic is library following latest bleeding edge Python development for validating data types in classes, a bit like those columns in rows of Excel data. It currently changes around regularly as this part of Python language is developing.

[10] Django is a framework for writing web apps I used for best part of past 16 years, it’s a well documented solid industry standard running some of the world's biggest sites and apps like Instagram and Pinterest. It had those things aforementioned Pydantic and its relations is solving worked out when some of its users were still in the nappies.