Neural Network powered Assembled PCB inspection tool (AOI)

[jpedroc]

Hi all,
     We have developed a neural network capable of detecting faults on the electronics components placed on a PCB just from a photo.
 The service has been released a month ago: www.AgnosPCB.com

Please feel free to take a look at our site and the current visual inspection tool: the web App https://app.agnospcb.com/ (You just need to drag and drop the photos you have taken to your PCBa)

This neural network powered inspection tool can detect missing/misplaced or rotated elements on any PCBA. It does not matter the type of elements placed/ PCB architecture/ color or size of the components.




How does it work?:
 The neural network uses a reference PCBA photo (a PCB that has been previously checked) and it will compare that reference with all the photos of PCBAs to be inspected.

The visual inspection takes seconds (from 2 to 5). Any mid-range smartphone can be used to take the photos (>8 Megapixels).

This fast visual recognition tool is extremely useful for low or medium quantity PCBAs batches

If you were interested in trying our APP, we will happily provide a USER ID / PASSWORD for free to inspect boards. There is no equipment investment required. Just your smartphone camera: https://www.agnospcb.com/how-to-use-your-camera-as-aoi
NOTE: For heavy workload, we have designed the inspection platform: https://www.agnospcb.com/agnospcb-inspection-platform

Anyone interested on trying the neural network, please contact me at: info@agnospcb.com or here https://www.agnospcb.com/contact

John C

[ThomasRR]

This is extremely interesting. We manufacture a lot of "small number batches" and a tool like this will save us a lot of problems (if works fine). Count me in
Email sent. How many "test inspection" are you providing?
Can I use my iphone for the test stage?
Thanks
Thomas

[jpedroc]

Hi Thomas.
 Sure. User account details sent.  
In the email, you will find some tips about how to use your smartphone to take the photos.
Just avoid light reflections on the PCB´s surface

[mikeselectricstuff]

I've always wondered how effective a simple flicker comparator would work - alternate the screen view between a reference image and live camera view, with some simple difference enhancement.
Should be doable on a RasPi pretty easily.

[Daixiwen]

This looks interesting.
I have some concerns about confidentiality though. What do you do with the pictures after they have been processed? Are they stored on your server? Who would have access to them?

[jpedroc]

We keep them in the server for 24 hours so a technician can check if the customer is making the most of the service (the photos is ok,  no reflections...etc) After that, the photos are automatically deleted.
Even when a client ask us to keep the files in our cloud server (for different reasons), storing the files is not an option (we currently have >40 clients and some of them are taking up to 12.000 images per day)

[jpedroc]

Comparing pixels/images does not work. The solders are always different for every SMD elements. The same components, on differente boards, can present some degree of variation (rotation, location, identification text...) and a simple image comparison will highlight a huge amount of "false errors"
Check this image: the same element from different boards. The solders present differences but the neural network knows that it is not a fault at all

This one shows potentiometers with different "screws" adjustment. A pixel comparison will indicate a "problem" when it is not the case.

This is how the neural network works. Uses a reference board as "golden sample" and check if the consecutive boards presents errors comparing like a human would do, the boards.


The main advantages of a system like this are its tolerance and speed. A regular AOI/ FAI system may tag some variations among elements as "faults" when they are not. For the example above. The neural network needed 1.8 seconds to analyze  the board.

[mikeselectricstuff]

We keep them in the server for 24 hours so a technician can check if the customer is making the most of the service (the photos is ok,  no reflections...etc) After that, the photos are automatically deleted.
Even when a client ask us to keep the files in our cloud server (for different reasons), storing the files is not an option (we currently have >40 clients and some of them are taking up to 12.000 images per day)

Maybe there is scope for an additional paid service of storing images for traceability?

[jpedroc]

We do not offer that (for now): The result of the inspection (the image) can be downloaded by the client 

[ThomasRR]

I just wanted to state this here: this is the most impressive and helpful tool I have used in the last 10 years.
We are a small/ medium size assembly PCB company located in UK. We have two AOI systems here, but the thing that programming them to check the integrity of the boards we manufacture requires a technician for hours. After that, manually inserting the boards (many times splitting the wafers) in the AOI machines, takes forever.

That, "forced" us to, sometimes, discard low/medium batch orders or increase the total cost for the clients.

We decided to test this neural network service and... wow.
Check it by yourself.
Image 1: the board as it should be

image 2: the board with some elements we know are common problems when assembling components: short-circuit pins, a misplaced capacitor and a "tombstone"
image 3: the location of the faults. Processing time according to the web: : 2.5s I think is a bit more (maybe 4 seconds) but it could be the time uploading the image to the server, not processing.


It is impressive. A skilled technician needs about 40-120 seconds to look for a mistake and many times not finding it/all of them

For this test:
 Camera: Iphone 8 camera
 Test board: Old Ethernet card (150x95mm)

dank je agnospcb!
Thomas

[tszaboo]

Out of curiosity: How do you train the network?
AFAIK, Neural networks training work by giving a bunch of known good and known bad data before it works. Do I need to make a 1000 pictures for the training?

[SilverSolder]

This looks like a super cool idea.

Do you know what the approximate rate of false positives / false negatives is?

[jpedroc]

Out of curiosity: How do you train the network?
AFAIK, Neural networks training work by giving a bunch of known good and known bad data before it works. Do I need to make a 1000 pictures for the training?

We have been training the network taking photos of "faulty" and "good" elements the last 13 months. Creating a catalog of more than 45.000 errors you can find on a PCBA.
But keeping in mind details like: slightly rotated capacitor does not mean that the component is "faulty" but a diode placed backwards (but aligned) is a big nope. Different texts on top of certain components (modules) may vary from batch to batch but that difference is not important.
Big/vertical components like electrolytic capacitors/ potentiometers / coils can be tilted but that is not a problem neither...and so on.

Right now, you just need to take one photo of a reference board and then to the boards you want to compare it with. It is an extremely fast process. The neural network will highlight where the faults are as a human would do (not paying attention, for example, to differences in the solder points...image below)

[jpedroc]

This looks like a super cool idea.

Do you know what the approximate rate of false positives / false negatives is?

Thanks.
Right now, the rate is: 99.987% of faulty elements detected.
We can know that number as we have been visually inspecting all the boards we have processed (every single one) and noted all the faults we have detected
Is the case of doubt, the suspicious element is highlighted so the technician can double-check it

[Calder]

Really interested. Can I have a test account please?

[jpedroc]

Message with account details sent. Everything about how to use it and tips included in the email ;-)

[jpedroc]

For everyone PM me about how it really works. Please check this video https://video.wixstatic.com/video/87e1b6_24db3d3e9c8147b8b8638a82f59ea0a3/1080p/mp4/file.mp4 (of the inspection tool we have just released). It shows some of the features and the way the service works.
Have a nice weekend!

[ElectronRob]

In your example pictures the software didn't seem to pick up the Atmel IC mostly off its pads... 6th picture down. Why is that?

Can users input data to be trained or do you only use your trained data?

How well does it detect solder bridging on leaded IC's? reliably?

Have you used this post reflow, but after placement?

What is the highest res image you can accept. For example if I take 10 close up images using a camera on an XY stage and stitch them all together you should be able to make out component markings very clearly - would the software be able to compare part numbers between components?

Great project, and congratulations on your progress so far.

Rob

[jpedroc]

The example images were created "filtering" other faults. So only some kind of errors were displayed. But thanks for the heads up. Good catch.

Every time a user uses the Agnospcb neural network it learns from its mistakes. We have staff continuously reviewing the results and giving guidance to the AI. So,in a way, yes: the users are training indirectly the neural network, and not really doing anything by themselves.

Solder bridging detection: Very well. But I have to stress this: the quality of the detection will heavily depend on the quality of the photo. The neural network can not see on a photograph what a skilled technician can not see. But it behaves in the same way: the neural network is "tolerant" when it finds a slightly rotated capacitor (but in "working condition"), some minor volume soldering differences...etc. But it knows when a defect has to be highlighted. This is an advantage vs regular AOI systems.

The system can handle up to 24Mpixels images without problem. And right now, that you are proposing about stitching images will work like this:
1) The image of the REFERENCE board can be as large as you want
2) You can send (without stitching, that is the good thing) a photo of a certain area of the board you want to inspect. The AI will know where this image belongs to and will check for faults comparing REFERENCE BOARD and that section of the BOARD TO BE INSPECTED. Check the image attached. The REFERENCE is 32 Mpixels, the cropped image, just 3500x3900 pixels. The Inspection tool software can handle that without problems and can even BATCH processing all the images.
 inspection of a small PCB area (no spatial references needed).jpg
Thanks, Rob. We really appreciate your words
If you want to test the service by yourself, please email me to info@agnospcb.com

" alt="" class="bbc_img" />

NOTE: We are about to release the Inspection tool Software: https://www.agnospcb.com/inspection-software-user-manual. It was a request from our customers. You can take a look to its features following the link (Working on it).

[ElectronRob]

Thank you for the reply.

May I suggest the addition of a QR (or one of the machine vision alternatives) reading feature. I would suggest that your batch processing would be better with some individual way of identifying a board within that batch. This would allow you to export a report against a specific PCB number etc.

I like it a lot, I have had a little play with this 'gold standard comparison' without any thing clever like a neural Network or ML, I find I can identify differences between boards well, I can ignore certain differences by permitting a component skew (I actually use a bounding box and if the difference is outside of the bounding box I flag the error) however I have always struggled with the no component placed, and tomb stone etc failure cases. Detecting nothing I have found very challenging! 

I think your solution presents a great option. Especially between machine and oven where misplaced components or no place is an issue when using lesser machines. I need to have a sit down and work over the pricing but would love to give it a go. Any plans to make this stand alone in the future? Something that could be run on a customers server rather than your own?

Good luck

Rob

[jpedroc]

Rob, thanks for the feedback.
We are working on the QR code automatic recognition. The board identification can be solved, right now , by checking the images of the BATCH processed using the inspection software. Checking the images, you can see the identification label/ QR code and the faults detected (if any) on that board.

All the inspected board results are locally stored, so the technician can get back to the images anytime.

Stand-alone version: what makes this approach so powerful is the "Cloud" neural network that is continuously learning. So, the local processing option has been discarded as an alternative. The other thing that make us opted for the online solution: the machines in charge of processing the images within seconds are... extremely powerful. A client will have to spend >2000 USD/computer to get the same results.

Contact me if you want to give it a go for free ;-)

[mikeselectricstuff]

This might be a useful place for you to post some info on your service : https://www.smtnet.com/Forums/

[jpedroc]

This might be a useful place for you to post some info on your service : https://www.smtnet.com/Forums/

Hey Mike... thanks! 

[JonasUK]

Crossed post from another sub-forum. But I just wanted to give an update here:

Well...We have been using the Agnospcb service a lot for the last 10 days (more than 550 boards inspected). And finally... we have adopted it.
We were skeptical about it, but it works very well (although not at first). It is fast and more important, reliable.

We had problems taking good pictures of the PCBAs during the trial phase (we are not photographers), but between the help of the Agnospcb staff and the fact that we finally switched to using a good camera (canon m200), the change has been spectacular. They offer an "official" inspection platform, but with our "DIY installation" in the factory - the mentioned camera and good and uniform light- we are just fine. We'll see if it's enough, but for now, it's working very well.

I have to point this out: they should not indicate that using a smartphone's camera during the trial stage can be an option. The difference a good camera makes is huge when talking about detecting faults. It is impressive how their system can work with low quality images, but a regular camera may lead to not appreciate the real capabilities of this AOI.

In short: for now Agnospcb (and its potential) is the winner for the small and fast runs we have in production. We have even tested it during the pre-oven stage, where the components are placed on top of the solder paste, and it worked! The missing / misplaced elements are a pain in the ass at this stage but all the defects were highlighted.

Agnospcb support told us that that was not the intended use (but that they are working on it and a new upgrade for the service will be released soon with this capability). From my point of view... it is already working.

If we find any important other point to be mentioned, I will post it in this forum.

I have attached Agnospcb service detected faults examples. Screen captures from their inspection tool software. Top square image: reference board, middle: location of the fault. Bottom: the inspected board.
 The thing is:  minor geometrical differences are not taking into account, but the important defects are. Check the drill holes and the silk layer. They are never the same among PCBs, and of course, they are not pinpointed as faults. 

[kristinapetrova]

Hey everyone, I've been considering using this tool for my PCB assembly process and I'm curious to hear about your experiences with it. Has it been reliable in detecting faults? How easy is it to use, especially when it comes to taking the photos for the inspection?

I'd really appreciate any feedback or insights you could share. Thanks in advance!