The ask
A Silicon Valley chip manufacturer was looking for a new way to identify and remove cracked silicon dies (small blocks of semiconductor material) from the production line in real time, without disrupting the manufacturing process.
Historically, these cracks were easy to spot using low grade vision systems. But as the requirement for smaller chips increased, die manufacturers have had to reduce wafer thickness via backgrinding. Backgrinding produces lines (a.k.a. scars) on the dies that make cracks harder to identify.
The manufacturer asked for our help in creating a new solution that could be embedded into their current production line and related systems, and that could meet three business acceptance criteria:
What we did
We created an A.I. image recognition solution that uses existing and
synthesized data to accurately classify dies as “GOOD” or “CRACKED”. The
solution then appropriately instructs the production line to continue or
remove each die.
Seamless integration with the client’s existing systems ensures this is
achieved while maintaining the third criteria: production speed.
Thanks to the use of Machine Learning, the solution is fully retrainable,
meaning its accuracy is continuously and remotely improved. This also
means it has the potential to be retrained for multiple different product
types.
A.I. IMAGE RECOGNITION IS ABLE TO ACCURATELY CLASSIFY DIES AS ”GOOD” OR ”CRACKED” WITHOUT IMPACTING PRODUCTION SPEED.
The process
There were three challenges facing quality assurance.
CHALLENGE 01
PROVIDING ACCURATE A.I. TRAINING DATA
The first challenge was a lack of data available to train the A.I. with,
which is a common issue for anomaly detection projects. In this case,
there were very few numbers of “CRACKED” die images to work with. We
used two data augmentation approaches to solve this. The first was to
split each cracked image into smaller windows, and then flip and rotate
them to provide 10x the amount of examples. We also used data
synthesization techniques to create hugely realistic replicas of cracked
dies using real world examples as a base. It was crucial in this stage to
ensure that the A.I. model was able to detect real-world cracks by using
information from the synthesised cracks – in other words, they needed to
be realistic enough to give an accurate reading. Once we had verified that
the models were accurate enough, we were able to combine these two
methods to give us a representative amount of information to train the
Machine Learning model, meaning we could accurately classify a wider
variance of images.
01
Original
02
Window Creation
03
10x Augmentation
CHALLENGE 02
DIFFERENTIATING CRACKS FROM SCARS
Our second challenge was to enable the A.I. solution to differentiate
cracks from backgrinding scars. Unlike cracks, the scars occur in
uniformed patterns due to the backgrinding process. Through careful
research and experimentation, our experienced Data Scientists were able
to identify the best scientific approach of using Fourier Transforms to
identify backgrinding stripe masks. We then applied these masks to the
images and then reversed the Fourier Transforms to provide cleaned
images. Since cracks are never uniform, this process ensured cracks
would not be removed from images by mistake.
01
Original
02
Masking scars
03
Cracks become visible
CHALLENGE 03
MEETING COMMERCIAL DEMANDS
The final challenge was to meet the production line requirements, allowing
the solution to work not just from a theoretical point of view, but to be
applicable in the real world where it can meet the commercial demands of
the business. To achieve this, we needed to take A.I. out of the lab and
into production by using our 20 years’ engineering and integration
experience and understanding of technical capabilities gained from
Intelygenz Labs. In this case, we needed to utilise the customer’s
centrally-controlled Edge computing abilities in order to meet the
operational needs and ensure commercial viability. To achieve this, we
used a cloud environment to simulate and fine tune the solution, with the
intention of porting it to local hardware once ready. This approach allows
us to flexibly make adjustments and ensures the final solution will meet
the speed requirements when applied in the real-world environment.
In the current simulated environment, this computer vision A.I. solution
automates the identification and diversion of faulty dies with an extremely
high level of accuracy. The solution has a false negative error rate of 0%,
meaning no faulty dies are classified as “GOOD”.
THE FINAL SOLUTION CLASSIFIES DIES WITH A FALSE NEGATIVE ERROR RATE OF 0% AND FALSE POSITIVE ERROR RATE OF < 1%.
We’ve also significantly reduced the false positive error rate that was
originally above 12% to below <1%, meaning very few non-faulty dies are
classified as “CRACKED”.
Current speed tests indicate images can be processed in less than 0.7
seconds, which we are able to improve by fine tuning and removing
complexities from the A.I. model. Once we achieve the business
acceptance criteria of less than 0.4 seconds while maintaining the same
false negative and false positive error rate, the solution will be placed into
full production.
Further improvements
Though we built this image recognition software for a specific product, its
A.I. model is continuously trainable. This means as more “GOOD” and
“CRACKED” dies are identified, the A.I. can continually learn from the data
and improve its accuracy over time. It also means the model can be
applied to any new products in the future, allowing the client to achieve
the same benefits of automation in manufacturing for other production
lines.
This solution can be applied to the quality assurance processes across
many industry verticals, as these same techniques can be trained to spot
imperfections in products on production lines throughout all stages of the
manufacturing process.
