Overcoming High Precision Optical Challenges
Overcoming High Precision Optical Challenges Using High-Speed 3D Inspection Technology: An Interview with Tim Skunes
Interview questions by Will Souter, AzoNetworks.
WS: Could you explain the basic principle which underpins how most 3D inspection technology systems work?
TS: There are a number of important 3D optical inspection technologies including white light interferometry, confocal scanning, and structured light triangulation.
CyberOptics industrial inspection systems typically use structured light triangulation where a series of structured light patterns are projected onto a measurement area and then imaged by high speed cameras that view the measurement area from a different direction. The imaged patterns are distorted by the objects being measured and the principles of phase shift profilometry can be used to accurately calculate the three dimensional shape of the objects from the distorted patterns.
For example, a fringe pattern is obliquely projected onto a measurement area in Figure 1 containing a rectangular block. The projected light is diffusely reflected and the pattern is then imaged by a camera from a different direction than the projector. The fringe pattern image is distorted by the height of the rectangular block as shown in Figure 2.
Phase shift profilometry has an unparalleled combination of speed and accuracy and, when architected properly, permits sub-micron measurement accuracy and inspection rates in excess of 50 million three dimensional measurement points per second.
Figure 1. Fringe pattern projected onto measurement area
Figure 2. Fringe image distortion
WS: Could you outline the main inspection challenges which exist when inspecting SMT assemblies?
TS: Many inspection challenges exist when inspecting SMT assemblies due to the varied nature of SMT applications as well as inspection processes and quality requirements. Some of the most common obstacles to high quality 3D inspection are tall components, highly reflective surfaces, modeling complex three-dimensional shapes and inspection speed requirements.
WS: How do tall components cause shadowing effects and why is this a problem for 3D sensing systems? What are the other challenges caused by tall components?
Many SMT assemblies, especially in automotive and industrial environments, contain capacitors and connectors which can be over 30mm tall. Triangulation sensing angles needed to provide adequate height accuracy will have blind areas on the circuit board near these tall components. In such scenarios, the shadow effect is a real concern as illustrated in the figure below.
Figure 3. Shadowed component
As it is sometimes required to inspect near the base of tall components, the ability to overcome “blinds spots” becomes critical.
The projection of high frequency patterns is required to meet inspection accuracy requirements for small components and features such as solder joints. However, the high frequency patterns go out of focus quickly and are completely blurred out on tall components. So several additional, coarser frequency patterns are frequently projected to measure the wide range of feature sizes.
WS: Why do reflective surfaces pose a challenge when inspecting SMT assemblies?
TS: Highly reflective surfaces present several challenges for SMT inspection. For example, the mirror-like surfaces of solder joints can generate glints which in turn cause camera saturation. Other mirror-like surfaces may direct light predominantly away from the camera view which in turn produces an extremely poor signal-to-noise ratio. Highly reflective surfaces also increase the severity and probability of multiple reflections. An example is shown in Figure 4.
Figure 4. Multiple reflections between highly reflective solder joints
WS: Could you explain how sensor architecture is crucial in overcoming inspection challenges related to multiple reflections?
TS: Multiple reflections disturb the fringe patterns and cause erroneous height measurements unless their effects can be suppressed. The fringe pattern observed at measurement point A, for example, shown in Figure 5 receives direct illumination from the projected blue pattern and indirect illumination from light scattered at point B. The indirect illumination arriving at point A disturbs the fringe pattern image.
Figure 5. Multiple reflections
A 3D phase shift profilometry system is able to suppress potential height measurement errors induced by multiple reflections by careful analysis of the image data sets from multiple cameras and multiple fringe pattern frequencies and directions.
Both low and high frequency patterns are required for tall components and connectors (e.g. 25 mm tall) in order to unwrap the fringe pattern phase and determine absolute height. However, the multi-path reflections from lower frequency patterns can severely corrupt the reported phase as shown in Figure 6.
Here, a multiple reflection adds coherently to the direct reflection. The potential height error is shown and is influenced by the relative strengths of the direct reflection and the multi-path reflection. These effects are also more severe for closely spaced components and highly reflective features.
Figure 6. Multi-path reflections of a lower frequency pattern
In general, data from locations of high fringe contrast are most reliable when there are no multi-path reflections. However, multi-path effects from lower frequency patterns also have high fringe contrast, but can severely corrupt the reported phase as seen in Figure 6.
Multi-path reflections of a high frequency fringe pattern is shown in Figure 7. The multiple reflections from many locations add incoherently to reduce the fringe contrast, but disturb the phase very little.
Figure 7. Multi-path reflections of a higher frequency pattern
Since each oblique view camera will be affected by a multiple reflection differently, the individually reported heights from different oblique cameras will differ for the location of a multiple reflection. This fact can be used during analysis to identify areas of multiple reflections.
To suppress the effects of multiple reflections, a rich data set of the fringe contrast, the effective surface reflectance, and calculated phase are analyzed for each frequency pattern from each of the oblique view cameras. This data is combined, or fused together, to suppress the disturbances of severe multi-path reflections.
WS: How does parallel 3D sensing differ from single 3D sensing? How can parallel 3D sensing be used to increase image acquisition speed?
TS: The two factors that typically limit inspection speed with the 3D phase shift profilometry method are image acquisition time and the amount of time to process the acquired images. For example, one typical 3D phase shift profilometry architecture includes a single camera and multiple projectors to address shadows from tall components.
A common approach with this architecture is to sequentially acquire 32 or more images of different fringe patterns with multiple frequencies and from four projection directions.
To address the requirement for improved acquisition speed, the 3D phase shift profilometry system can be architected with a single fringe projector and multiple oblique viewing cameras (as shown in Figure 8) enabling image acquisition of a particular fringe pattern simultaneously. This results in a degree of parallelization in the image acquisition with a typical 4X reduction in image acquisition time.
Figure 8. Sensor architecture
Also, the entire 3D field of view is completely illuminated by the vertical projector with no areas of shadow.The projected fringes are then imaged simultaneously by the oblique view cameras as illustrated in Figure 9.
Figure 9. Vertical projection with oblique view cameras
Although there is a tremendous speed advantage to the multiple camera architecture, it can be seen from Figure 8 that the X,Y coordinates for a given camera pixel are a function of the Z height.
To overcome this challenge, the 3D phase shift profilometry system must accurately calibrate the X,Y locations for every pixel in every camera throughout the entire Z measurement range. This essentially amounts to calibrating the ray slope for every camera pixel. If the calibration is performed accurately, then the X, Y locations may be unambiguously decoded to micron level accuracies.
You can find the full interview with downloadable white papers on AZO’s website.