Author Topic: Are there any DIY Automated Optical Inspection (AOI) solutions available?  (Read 20404 times)

0 Members and 1 Guest are viewing this topic.

Offline protoneerTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: nz
I have been building lots of boards lately and I was wondering if there are any DIY / Opensource Optical Inspections libraries or software available?

Most of the time I did manual pick and placing but I am moving to machine placing and it opens me up for to making more mistakes with volume.

I found this blog that seemed very helpful...http://bobbaddeley.com/2015/12/creating-an-automated-optical-inspector-for-50/


But this is only a small part of the picture and I don't want to re-invent the wheel but is willing to contribute towards a project like this.
 
The following users thanked this post: ataradov

Offline mrpackethead

  • Super Contributor
  • ***
  • Posts: 2845
  • Country: nz
  • D Size Cell
I was discussing this with Thommo, last night, and with the open-pnp guys..  Open-CV already provides a way to do pattern recongiition.

It would'nt take too much to run the downfacing camera of your pnp machine across the board. and do the checking?
On a quest to find increasingly complicated ways to blink things
 
The following users thanked this post: ChunkyPastaSauce, protoneer

Offline rx8pilot

  • Super Contributor
  • ***
  • Posts: 3634
  • Country: us
  • If you want more money, be more valuable.
Joining this discussion - hoping for the same thing.
Factory400 - the worlds smallest factory. https://www.youtube.com/c/Factory400
 

Offline ChunkyPastaSauce

  • Supporter
  • ****
  • Posts: 539
  • Country: 00
If you can provided a good training data for your application, then you may want to look into machine learning methods. I've had very good results with LIBSVM https://www.csie.ntu.edu.tw/~cjlin/libsvm/ in other applications, and it's fast  enough for many realtime applications after training; I haven't tried it for OCR but I'd expect the SVM method to work well for many OCR problems.
« Last Edit: March 30, 2016, 02:58:12 am by ChunkyPastaSauce »
 
The following users thanked this post: mrpackethead, protoneer

Offline mrpackethead

  • Super Contributor
  • ***
  • Posts: 2845
  • Country: nz
  • D Size Cell
nice nick. buy the way.
On a quest to find increasingly complicated ways to blink things
 

Offline ChunkyPastaSauce

  • Supporter
  • ****
  • Posts: 539
  • Country: 00
nice nick. buy the way.

Thanks

I just took a look at the open-cv libraries. It includes a bunch of machine learning algorithms, including a SVM implementation [actually it's LIBSVM lol]........awesome!
« Last Edit: March 30, 2016, 03:08:44 am by ChunkyPastaSauce »
 

Offline mrpackethead

  • Super Contributor
  • ***
  • Posts: 2845
  • Country: nz
  • D Size Cell
http://bobbaddeley.com/2015/12/creating-an-automated-optical-inspector-for-50/

"n the world of electronics assembly, parts are really small, quantities are really large, and equipment is really expensive. The process of putting all the components onto circuit boards thousands of times per hour with really high accuracy is challenging, but tools are getting easier and cheaper every day.

At my job (Quietyme), we’re doing our own circuit board assembly. We had to purchase some equipment, like a pick and place, which for $5000 automates the process of taking the small components off of reels and putting the components on the circuit board. We also have a stencil screenprinter and a reflow oven (toaster oven). Total investment so far has been under $6000.

The most recent tool to add to our arsenal is an Automated Optical Inspector. The purpose of the AOI is to examine the circuit board to find defects in the assembly process. Sometimes a component doesn’t solder on correctly, or there is a bridge where there shouldn’t be. These need to be found and fixed before they are programmed and used. We had been doing it manually with just visual inspection, but that was tedious and not very accurate, especially after the 100th board.
"
On a quest to find increasingly complicated ways to blink things
 

Offline Smallsmt

  • Frequent Contributor
  • **
  • Posts: 598
  • Country: de
Some months before I thought about to add this function to our PNP machine.
But there are some more complicated things to solve.
You need to add more than one camera from minimum 2 directions and light from different directions if you want to inspect the solder points too.
It's easy to detect if there is a part or not but solder point inspection is more complicated.
 

Offline thommo

  • Frequent Contributor
  • **
  • Posts: 268
  • Country: au
Congrats Nick
Nice progress.

As SmallSMT points out - no solder inspection, but every project starts somewhere and what you've achieved to date is all currently required to be done using the 'eye strain' method on every board anyway - so a big early step.

It seems most Cartesian machines don't populate max PCB area anyway, so mounting an additional camera on the head should solve the problem. It doesn't really matter how many views or board sections are required in order to cover the whole area. I don't think image stitching would help either because interpolation could add some artefacts to the mix.

We've dealt with a machine vision company called PointGrey previously and their product is great. With a good high resolution image, things like solder bridges should be detectable with high reliability also, given the footprint data already exists.

Well done!!!
 

Offline protoneerTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: nz
I was discussing this with Thommo, last night, and with the open-pnp guys..  Open-CV already provides a way to do pattern recongiition.

It would'nt take too much to run the downfacing camera of your pnp machine across the board. and do the checking?

That's a brilliant idea!!! I have a TVM802 arriving shortly and it sounds very hack-able as the cameras are just normal USB type and it looks like the Machine control comms have been hacked aswell.
 

Offline protoneerTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: nz
I am also interested in checking solder-paste before starting the run,so some kind of inspection would be better than nothing.

Importing component location data from an EDA(Kicad) might also be an option to filter out only the bits you are interested in.

 

Offline l0wside

  • Regular Contributor
  • *
  • Posts: 118
  • Country: de
Solder paste print should be pretty simple. The solder paste is grey, the pad is either silver or gold, and the solder mask is green.

Checking QFNs will be a pain.

Max
 

Offline mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 13695
  • Country: gb
    • Mike's Electric Stuff
I though of one way to do it super-cheap. Set up a decent camera with view of the whole PCB, with an alignment jig ( or a decent area), take a photo of a reference PCB, then place  the DUT in teh jig, take a pic and show the two pics alternately - the eye is avery good at spotting changes, so although not automatic, I think it could be very quick, quicker than 'real' AOI. You could add enhancement like emphasising changes, and a pan/zoom that tracks the two images.
Considering the high resolution of modern digital cameras I think it could be a viable approach.
 
 
Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 

Offline kayvee

  • Regular Contributor
  • *
  • Posts: 151
  • Country: za
A low cost solution would be a huge help in my setup too.  I have been looking around for something but commercial options are crazy expensive.

I've trained SWMBO on the task but it's not the most foolproof solution  :scared:

Keen to see where this goes...
« Last Edit: March 31, 2016, 01:59:34 pm by kayvee »
 

Offline l0wside

  • Regular Contributor
  • *
  • Posts: 118
  • Country: de
Not having the slightest clue about the inner works of image processing, I just played around with an Android demo: http://www.barrythomas.co.uk/machinevision.html

The "Colour Range" function is quite simple. After setting up Python/OpenCV, finding a PCB in an image should be possible during an evening of playing around with a webcam and a computer. Finding all grey areas and checking if they match with the stencil definition should not pose a major problem either. The biggest effort might be comparing the Gerber data to all these rectangles that OpenCV will return.


Complete solder joints are a different story. Professional systems usually do not look from straight above, but rather at 45°. Otherwise, you could not detect a solder joint where a TQFP pin dangles above the pad.

Max
 

Offline johnwa

  • Frequent Contributor
  • **
  • Posts: 255
  • Country: au
    • loopgain.net - a few of my projects
I had a bit of a play around with this before using ImageMagick to find the difference between the sample and reference images. This was primarily targetting missing components, not bad soldering, I had good results transforming to HSV colour space, and just examining the saturation channel, though a better approach would be to use some function of all three channels. The saturation channel is colourised to red, and 'eroded', to reduce the noisiness, then blended back into a darkened version of the original.

I didn't get round to building a proper light box and alignment jig, but results were promising just using a desk lamp and a the camera in a smart phone that was resting on a stack of books.

I used the following shell script, if anyone wants to experiment:


#!/bin/sh

REFERENCE="20160118_111042.jpg"
SAMPLE="20160118_111108.jpg"

# Using BMP for these doesn't seem to give the same result, colour depth/gamma issues?
convert $REFERENCE  -colorspace HSV -separate /tmp/reference%d.pgm
convert $SAMPLE     -colorspace HSV -separate /tmp/sample%d.pgm

cd /tmp
composite reference1.pgm sample1.pgm -compose difference out.pgm
convert out.pgm -morphology Erode Square out2.pgm
convert out2.pgm -channel G -evaluate set 0 -channel B -evaluate set 0  out3.bmp
convert out3.bmp -brightness-contrast 40x50 out3.bmp

convert "$REFERENCE" -modulate 100,30 -brightness-contrast -30 reference-dark.bmp
cd -
composite /tmp/reference-dark.bmp /tmp/out3.bmp -compose plus out4.png


PGM was used for the intermediate files for speed, with the eventual intent that the process could be applied to a live video stream - put the board under the camera, and as soon at is registered correctly, the result would be displayed on the monitor. I think processing time was only a second or so for an image of a few megapixels; no doubt it could be accelerated further if desired.
 
The following users thanked this post: cdev

Offline Brumby

  • Supporter
  • ****
  • Posts: 12288
  • Country: au
Not sure if this is covered in the software being used - but ....

Whenever you take a photo, particularly at close range (less than a metre or so) you will get some degree of barrel distortion.  The longer the distance and/or the longer the focal length of the lens, the less it will be - but practical situations for something the size of a PCB (or panel) means it is a real consideration.

When you take a photo of a reference board and then, using the same optical setup, compare that to the board being examined, these distortions will be the same, allowing a fast and accurate examination - whether using a human (such as the 'blink comparator' method mentioned above) or machine vision.

If you then want to take your CAD quality data and map it to the image, the distortion will need to be corrected or at least accounted for, since the drawing files will have digital precision on perfectly rectangular coordinates - and the photo won't.
 

Offline JardCrocker

  • Newbie
  • Posts: 1
  • Country: us
Re: Are there any DIY Automated Optical Inspection (AOI) solutions available?
« Reply #17 on: November 02, 2017, 05:13:26 pm »
Hi..i am a new user here. As per my knowledge Open-CV already provides a way to do pattern recognition.It will take too much to run the downfacing camera of your pnp machine across the board.

turnkey pcb
« Last Edit: May 06, 2022, 08:19:00 pm by JardCrocker »
 

Offline mrpackethead

  • Super Contributor
  • ***
  • Posts: 2845
  • Country: nz
  • D Size Cell
Re: Are there any DIY Automated Optical Inspection (AOI) solutions available?
« Reply #18 on: November 02, 2017, 06:16:51 pm »
The commerical machines i've seen seem to run a X/Y gantry over boards with their cameras.  They also seem to use differnet color light for differnet process's.

If i only had time to spend on this, it would be awesome.    Compared to building a PNP machien though, mechanically this shoudl be easy!
On a quest to find increasingly complicated ways to blink things
 

Offline protoneerTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: nz
Re: Are there any DIY Automated Optical Inspection (AOI) solutions available?
« Reply #19 on: November 02, 2017, 06:52:48 pm »
The commerical machines i've seen seem to run a X/Y gantry over boards with their cameras.  They also seem to use differnet color light for differnet process's.

If i only had time to spend on this, it would be awesome.    Compared to building a PNP machien though, mechanically this shoudl be easy!

I am slowly starting to look at a project doing this. The hardware for movement is simple. Consistent lighting and camera is a huge factor.

Software will be the magic sauce but can only be looked at as soon as the other stuff can be done consistently.
 
The following users thanked this post: cdev

Offline protoneerTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: nz
Re: Are there any DIY Automated Optical Inspection (AOI) solutions available?
« Reply #20 on: November 02, 2017, 06:56:25 pm »
One good thing about running a small company is that resources are limited and it forces you to innovate.

Funny that Quality and Inventory management is still my major issues...  :-DD
 
The following users thanked this post: cdev

Offline mk_

  • Regular Contributor
  • *
  • Posts: 225
  • Country: at
Re: Are there any DIY Automated Optical Inspection (AOI) solutions available?
« Reply #21 on: November 02, 2017, 07:36:28 pm »
The commerical machines i've seen seem to run a X/Y gantry over boards with their cameras.  They also seem to use differnet color light for differnet process's.

If i only had time to spend on this, it would be awesome.    Compared to building a PNP machien though, mechanically this shoudl be easy!

Software will be the magic sauce but can only be looked at as soon as the other stuff can be done consistently.

https://www.mevislab.de/

This is sw written for - at the first sight - completly different field but - comparing pictures in a medical disipline isn`t very different from good/bad placement of a pcb...

there is also a free version aviable with some minor restrictions, which shouldn`t harm a solution for pnp-verification, see https://www.mevislab.de/mevislab/versions-and-licensing/

hth

michael
 
 

Offline cdev

  • Super Contributor
  • ***
  • !
  • Posts: 7350
  • Country: 00
Re: Are there any DIY Automated Optical Inspection (AOI) solutions available?
« Reply #22 on: November 02, 2017, 08:54:54 pm »
The concept of eigenvalues and eigenvectors - related terms (for example, eigenimage, eigenface) may bring you to a fair amount of stuff on differences from average values.

I also remember having seen writeups of projects using various machine vision toolkits (there are quite a few tools out there) Opencv is just the best known of many-  etc. to do electronics quality control.

For example, you might want to look at the world of GIS tools. Your application is actually a GIS application, its just the area you are looking at is smaller than most of them usually are.

I had a bit of a play around with this before using ImageMagick to find the difference between the sample and reference images. This was primarily targetting missing components, not bad soldering, I had good results transforming to HSV colour space, and just examining the saturation channel, though a better approach would be to use some function of all three channels. The saturation channel is colourised to red, and 'eroded', to reduce the noisiness, then blended back into a darkened version of the original.

I didn't get round to building a proper light box and alignment jig, but results were promising just using a desk lamp and a the camera in a smart phone that was resting on a stack of books.

I used the following shell script, if anyone wants to experiment:


#!/bin/sh

REFERENCE="20160118_111042.jpg"
SAMPLE="20160118_111108.jpg"

# Using BMP for these doesn't seem to give the same result, colour depth/gamma issues?
convert $REFERENCE  -colorspace HSV -separate /tmp/reference%d.pgm
convert $SAMPLE     -colorspace HSV -separate /tmp/sample%d.pgm

cd /tmp
composite reference1.pgm sample1.pgm -compose difference out.pgm
convert out.pgm -morphology Erode Square out2.pgm
convert out2.pgm -channel G -evaluate set 0 -channel B -evaluate set 0  out3.bmp
convert out3.bmp -brightness-contrast 40x50 out3.bmp

convert "$REFERENCE" -modulate 100,30 -brightness-contrast -30 reference-dark.bmp
cd -
composite /tmp/reference-dark.bmp /tmp/out3.bmp -compose plus out4.png


PGM was used for the intermediate files for speed, with the eventual intent that the process could be applied to a live video stream - put the board under the camera, and as soon at is registered correctly, the result would be displayed on the monitor. I think processing time was only a second or so for an image of a few megapixels; no doubt it could be accelerated further if desired.

PGM is actually used as the intermediate file format because thats its purpose, for use in applications exactly like the one you describe.
« Last Edit: November 02, 2017, 09:25:24 pm by cdev »
"What the large print giveth, the small print taketh away."
 

Offline mrpackethead

  • Super Contributor
  • ***
  • Posts: 2845
  • Country: nz
  • D Size Cell
Re: Are there any DIY Automated Optical Inspection (AOI) solutions available?
« Reply #23 on: November 02, 2017, 09:22:55 pm »
have you got some pictures you can post? of the input/reference output of those?
On a quest to find increasingly complicated ways to blink things
 

Offline cdev

  • Super Contributor
  • ***
  • !
  • Posts: 7350
  • Country: 00
Re: Are there any DIY Automated Optical Inspection (AOI) solutions available?
« Reply #24 on: November 02, 2017, 09:23:14 pm »
Look at GIS software and software for interpreting space imaging. Lots of it is free. For example, the ESA has a bunch of tools, fairly advanced ones which can give you quantitative analysis from bitmaps - they are designed for automated analysis of things like vegetation, terrain cover, changes in geographical things like roads and waterways.. that maps very closely to circuits, from the point of view of the software.

Another area is segmentation software. There look at medical imaging software. If any of you have a Mac, check out Osirix, which is for medical imaging, you can perhaps get some ideas on what those kinds of terms mean. For example, you might have certain areas on your boards where the same errors always come up. Likely you could hack together some automated way of examining them using this wealth of free software.

One good free resource is the NASA Technical Reports Server.

Try a search on machine vision in the context of electronics quality and process control.

There is tons of useful scientific information on things like that on NTRS. There is a good chance you could find something specific to your problem domain that you did not need to adapt.
« Last Edit: November 02, 2017, 09:27:27 pm by cdev »
"What the large print giveth, the small print taketh away."
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf