How to tell who's the "best engineer"?

[eecook]

Hi All,

This is a mental exercise for the most experienced EE designers that might help novices like myself, so bare with me please.
Let's say you are faced with the following problem. You are working on a project and you need to bring a senior design engineer to the team with exceptional well-rounded electrical engineering skills. It is critical for the project success that the ideal candidate is found.
But there are a couple of problems:
  - There are 100+ candidates
  - You are not allowed to meet the engineers in person
  - Their resumes are not available to you
  - There is only one chance to hire, so you must get it right the first time or you'll likely kill the project
  - There are infinite enough (see EDIT #7) resources to test the candidates.

The members of the team come up with a way of selecting the best candidate. Their approach is to find a practical problem that requires well-rounded engineering design skills, have all of the engineers submit their prototypes and test them against a predefined metric (i.e. whoever makes the fastest oscillator at X amount of power, jitter, etc)

[EDIT #1: IP is not a problem. The problem put forth by the team is in the public domain, as well as the proposed solutions by the candidates]
[EDIT #2: All the 100+ candidates have excellent social skills and will perform great within the team. Only the technical aspect needs to be assessed.]
[EDIT #4: All the 100+ candidates will go through the process willingly.]
[EDIT #5: The candidates have a time constraint to submit their prototypes.]
[EDIT #6: The candidates are informed about the evaluation metric.]
[EDIT #7: The team has already solved the problem in the past and have established what the right amount of resources is.]

***This is the most important piece of this game:***
It is a completely fabricated scenario, but the purpose and my interest here is to know what could the practical problems be, and which metric would be relevant in each case.
In other words, upon solving which practical problems would you be able to conclude that an engineer has an exceptional grasp of whatever he/she claims to know. From those problems, which have the special feature of being easily evaluated in terms of a simple set of metrics (like the example of the oscillator).

[EDIT #8. Here's an example of an answer I would expect:
Note: I am not saying this is a good example
Basic Problem: Design and build a 240W solar grid-tie inverter that fits an altoids box.
Constraints:
    Regulatory: Must comply with UL 609050, UL1741, IEEE1547 and FCC Part 15, Class B.
    Input voltage/current: MPP per STC for 240W panel. SAS will be provided (Manufacturer X and model Y)
    Output: AC source and AC load (Manufacturer X and model Y)
    Misc:....
Evaluation metrics:
    CEC Efficiency and converter volume.
    The winning candidate will be the one whose design has the highest Efficiency/Volume ratio provided that all the constraints are met.
]

Thanks,

Simon.

[EDIT #3: This is a toy example and a completely ridiculous scenario. What matters to me when asking this is the practical problems that would emerge and the metrics to make the comparison. Don't worry about the context, the framework giving rise to the formulation of the problem could also be a contest like the "Little Box Challenge" or the "Tricorder X prize"]

[Bassman59]

The members of the team come up with a way of selecting the best candidate. Their approach is to find a practical problem that requires well-rounded engineering design skills, have all of the engineers submit their prototypes and test them against a predefined metric (i.e. whoever makes the fastest oscillator at X amount of power, jitter, etc)

What is this, "America's Next Top Engineer?"

This is how a fucking game show works, and it's why the US has a fucking game show charlatan host as President.

The "winner" of your contest is the person who says, "You want what when, and for free? Go fuck yourself."

[nctnico]

There are tools for that already: an aptitude test and a test to see if someone has relevant knowledge (like knowing a certain programming language).

[eecook]

There are tools for that already: an aptitude test and a test to see if someone has relevant knowledge (like knowing a certain programming language).

That's like testing who's the best MMA fighter with an aptitude test. Say you are walking by the street with a million dollars cash and need protection, you need more than the aptitude test, you wanna see the guy's actual capabilities and be damn sure about them. Let's assume that for our toy example the stakes of getting it right are really high as well.

[ChunkyPastaSauce]

Hi All,

This is a mental exercise for the most experienced EE designers that might help novices like myself, so bare with me please.
Let's say you are faced with the following problem. You are working on a project and you need to bring a senior design engineer to the team with exceptional well-rounded electrical engineering skills. It is critical for the project success that the ideal candidate is found.
But there are a couple of problems:
  - There are 100+ candidates
  - You are not allowed to meet the engineers in person
  - Their resumes are not available to you
  - There is only one chance to hire, so you must get it right the first time or you'll likely kill the project
  - There are infinite resources to test the candidates

The members of the team come up with a way of selecting the best candidate. Their approach is to find a practical problem that requires well-rounded engineering design skills, have all of the engineers submit their prototypes and test them against a predefined metric (i.e. whoever makes the fastest oscillator at X amount of power, jitter, etc)

You could do the all too common thing: assign them projects from the work that needs to be done

switching my answer because basically anyone competent and at actual senior level would:

The "winner" of your contest is the person who says, "You want what when, and for free? Go fuck yourself."

[rstofer]

The first practical problem is 'who owns the solution' and what happens when your company uses the IP owned by your contestant.

So, the problems can't be 'real', they must be contrived and of no commercial value.

To be clear, you expect candidates to spend their personal time solving a contrived problem.  Who would do that?  Well, the unemployed might, they have plenty of time on their hands.  But you don't want anybody who is unemployed - by definition.  The real expert is going to laugh their ass off and just shred the paperwork.

I don't think this game show approach will be a winner.

[JohnnyMalaria]

Put all the candidates in the room together. Define a challenge and stand back and let the games commence. Now you can watch to see how the candidates behave, who shows strong leadership, who won't help others etc.

[Gregg]

The answer is nobody willing to submit to such foolishness would first of all want to work for you and those that show up would fall into one or more of the following categories
•   Desperate for a job
•   Less qualified than you are attempting to discern
•   Hoping for a stepping stone to something better
•   Someone that leaves in disgust before the charade is over
The best of a sorry lot will still be a member of the sorry lot

[DimitriP]

"the only winning move is to not play"

[filssavi]

Let’s say for example you find the brightest engineer in the whole world, he is extremely clever, in his field, however he is bad tempered, extremely arrogant and pretends that everybody else does exactly what he thinks and tell them, and throws massive tantrums when this does not happen...

Now everyone else in your company jumps ship ASAP because the best engineer is impossible to work with (basically a complete asshole) now you find yourself with the star engineer and a bunch of spineless individuals with no idea of their own to speak off


That is why teamwork and soft skills are as important as technical ones, we are noi in the 1700 anymore a single guy can’t do everything and when you have more people it’s important for them to work well together

Basically if you have engineers working together, the results is better than just the sum of the individual parts, the opposite is true if they fight just not cooperate, since each one is not giving it’s 100%

[RoGeorge]

As an employer, you don't need the best engineer. You need an engineer good enough to do the job.

The whole engineering is not about achieving the best possible performance, but to find a good enough performance to cost ratio while spending only a limited amount of resources. With infinite resources, anyone can achieve anything.

Other said, engineering is all about finding a good enough compromise with the given available resources.
Same for hiring. A good enough engineer is good enough, you don't need the best one.

[eecook]

Put all the candidates in the room together. Define a challenge and stand back and let the games commence. Now you can watch to see how the candidates behave, who shows strong leadership, who won't help others etc.

Could be. However, the key question is which challenge/s you would define and which clearly predefined metric you would use to evaluate the results.

[tpowell1830]

I have seen this scenario before in my many job interviews. My reaction to this dog and pony show was to decline and walk away.

In order to find the qualified engineer, I agree with RoGeorge. For instance, you need to move some dirt and you have several choices of dirt moving machines. The first will move 2 tons per hour, the second one will move 20 tons per hour and the third one will move 30 tons per hour. The first dirt mover is $20 per hour, the second one is $75 per hour and the third one is $80 per hour. You need to move 100 tons in 5 hours, which one would you choose?

The same type of evaluation goes to the engineer that you hire. Most engineering managers can make this evaluation and select 2 or 3 for second round. Second round is meeting some of the senior engineers and chatting. Unless there is an overall mesh from one of the candidates, the engineering manager will pick based on previous qualifications. The only thing that could make you stand out is if you really meshed with one of the senior engineers on a level that placed the candidate above others, such as a hobby that demonstrates the desired traits.

Just my 2 cents...

EDIT:This is a real world scenario, basically the way it is normally done.

[T3sl4co1l]

I'm not doing a design problem unless you pay me for taking your test.

Sometimes, "projects" consist of asking prospective employees/contractors for methods, approaches and solutions, then not calling them back afterwards...

I'm smart enough not to fall for that.

Does that mean I passed the test?

Tim

[Mr. Scram]

Almost all off the interviewing shenanigans are self deceit anyway. It turns out to be hard to reliably predict how someone will do and inviting one to three people and trusting your gut turns out to be as valid as some convoluted multi stage process. Of course, the latter is easier to cover your arse with when you inevitably get it wrong sooner or later, which explains the success of companies selling all sorts of ass covering schemes.

[EEVblog]

The answer is nobody willing to submit to such foolishness would first of all want to work for you and those that show up would fall into one or more of the following categories
•   Desperate for a job
•   Less qualified than you are attempting to discern
•   Hoping for a stepping stone to something better
•   Someone that leaves in disgust before the charade is over
The best of a sorry lot will still be a member of the sorry lot

This.
You won't get "the best engineer" with this method simply because are not dumb enough to apply in such a way.

Is there any sort of practical point to any of this?

[JohnnyMalaria]

"who's the "best engineer"?"

Casey Jones (steamin' and a-rollin')

[thermistor-guy]

... It is critical for the project success that the ideal candidate is found.
...[EDIT #7: The team has already solved the problem in the past and have established what the right amount of resources is.

"Logical contradiction alert, Captain."
"Acknowledged. Set phasors on stun."

[DimitriP]

Is there any sort of practical point to any of this?

Yes!
The point seems to be to come up with a method/approach/solution for selecting the "best engineer" by having the candidates do the work instead of the hiring party.

As some have already stated  in a more or less verbose fashion:
"the only winning move is to not play"

[EEVblog]

[TerraHertz]

Relevant:

The REAL Answer To The Viral Chinese Math Problem "How Old Is The Captain?"

Real engineers must be able to recognize an impossible design spec, and respond appropriately. (Where 'appropriate' depends on the circumstance.)

Coming up with a single test for a 'best engineer', seems like one of those specs.

Also, the video is a nice demonstration that even today, many people cannot Logic.

[EEVblog]

Real engineers must be able to recognize an impossible design spec, and respond appropriately. (Where 'appropriate' depends on the circumstance.)

Reminds me of this:

[Brumby]

I have to agree with this...

The answer is nobody willing to submit to such foolishness would first of all want to work for you and those that show up would fall into one or more of the following categories
•   Desperate for a job
•   Less qualified than you are attempting to discern
•   Hoping for a stepping stone to something better
•   Someone that leaves in disgust before the charade is over
The best of a sorry lot will still be a member of the sorry lot

Not having access to each candidate and their résumé is a poor start.  Often the résumé can give more information than the candidate might realise.  Once I had one from a candidate who had left Uni less than two years prior, but had listed so much "experience" that it was impossible to obtain any real value from that in less than 10-15 years in the industry.  Also, not meeting the candidate face-to-face means you won't get any body language cues or a good sense of how they interact with people.  There's just so much that your proposed process could fail on.

One simple question:  How will you know that the solution presented is actually the work of the candidate?

[ivaylo]

The OP told us:

This is a mental exercise ... that might help novices like myself ...

Good thread, hope he/she learns. Hiring in engineering is shooting in the dark. Those who are good at it posess something most of us don’t. The biggest problem I have with the premises laid is his/her  “Edit #2”.

[eecook]

Ok guys this post has been an epic FAIL  . My only goal was to pinpoint particular designs and engineering problems I could work on to improve my own skills. As a bonus I was hoping there would be some metric to easily measure my success so as not to fool myself into a competence illusion.

That's it! The fabricated "hiring somebody" scenario played differently in my head, I though it would be a good way to start the conversation, but I can see now that it was a terrible idea. So please let's forget about it 

If you got to this point, that's all you need. If you feel like you might be able to give me a hand you can keep reading.

Some background. I live in Argentina and in particular, the town where I am, well... there aren't many EEs around or even related companies out there, so it is just not a possibility to build meaningful relationships with the community, simply because there isn't one. There's no elder with a beard to go ask technical questions to and in terms of buying equipment and components, the borders are virtually shut, so while possible, it is not easy nor cheap to buy stuff.

I have made some good valuable experience in the industry (4 years). Now I am freelancing and it is all well and good. The fact is that I want to improve, I feel the necessity to learn the craftsmanship and mastery of the art. I need guidance and I want to be effective in how I use my time. The need for a metric is merely a way to keep myself bounded and be able to see my mistakes, considering that there won't be any mentors around. I have thought about getting a job overseas but at the moment it is not an option, but I don't discard it in the future.

I have not yet found my niche within EE, so I would like to keep on exploring. I know Altium, I have done a lot of PCB design in my last job, including power electronics, so I think I have a solid grasp there. I know a couple of things about component reliability, DFM and DFT. I can do basic analog stuff with op-amps and transistors, and even build some small (tens of watts) switching power supply. I can use a simulator like NL5 with confidence. I have done some embedded C and I am playing with new micro-controllers. I even took an Embedded Systems course in Coursera recently that was really good. I also have some knowledge and have done some machine learning for music information retrieval and I am really good with Matlab. Linux is another tool I can operate proficiently.

As long as they can be troubleshooted with "basic" equipment (Oscilloscope, low-range Spectrum Analyser, LCR meter, multimeter, DC bench supply and a function generator), I would like to improve my analog design skills and embedded system skills.

With all that in mind, I am looking for some project's I could work on to really improve those skills.

EDIT: If in your experience, you consider that there are skills of general application, please share that too.