Android Question Counting Overlapping Objects in Images: App Development Challenge

[jroriz]

Hello everyone!

I'm developing an application that needs to count the number of glass plates in photos. The plates are rectangular, of equal size, and usually arranged similarly in all photos. However, the challenge is that some plates may overlap, as in the example image below, where there are two overlapping plates on the right side.

The idea is for the application to be able to count the plates with the highest possible accuracy, even when there is overlap. The plates will always be more or less in the same position and have the same width, which may be useful for algorithm development.

I would like to know if anyone has any suggestions or tips for dealing with this overlap problem and ensuring accuracy in counting. What computer vision techniques or libraries would be most suitable for this case?

I appreciate any help or guidance you can give me.

Thank you!

 

[emexes]

The glass below the label looks like it'd be pretty straightforward to count, but the reflections of the top third are going to wreak havok.

Can we assume that at least some of the glass will be without reflection, and that the operator will take another photo if the current photo has no unreflecting glass?

 

[emexes]

Lol I don't know how much image code you write, but in B4X I only do it maybe once or twice a year, and so this might be the blind leading the blind...

but this time we've got a helper:

https://x.com/i/grok/share/n8fofyzNxJRqIKvxbQ9EJqi70

and I've managed to get this far without the usual careering off into the bushes = things are looking good :

 

[emexes]

Next step is to run a scanning line from left to right and at various angles, see how well it picks up those sheet edges. Might be thwarted by the edges not appearing straight enough in the photos due to lens perspective distortion or whatever it's called, but we'll cross that bridge when we get to it.

Horribly inefficient but, hey, we're not processing live video (right?), and phase one in programming is to get it working.
(followed by phase two: get it working correctly, and phase three: get it working fast enough).

 

[emexes]

Phase one in the ball-park. On to phase two. What could possibly go rwong?

The orange trace is dark-to-bright edges, and the blue line is light-to-dark edges.

Thus for a thin light line separating two dark pieces of glass, the orange peak should lead the blue negative peak, and the horizontal separation between the peaks should be the average width of the (usually) thin light lines.

 

[emexes]

This is the result of only scanning the bottom quarter, away from the reflections. So maybe a new refinement is to scan the image in horizontal bands, and choose the band that gives the best result.

 

[udg]

OT: did you consider ocr on the printed label? There's a suspect #14 repeated a couple of times.. and that appears to be the number of visible mirrors.

 

[jroriz]

The glass below the label looks like it'd be pretty straightforward to count, but the reflections of the top third are going to wreak havok.

Can we assume that at least some of the glass will be without reflection, and that the operator will take another photo if the current photo has no unreflecting glass?

Well, considering that it’s a factory worker taking the photos, the image quality can vary a lot. Really, anything can happen, from reflections to poor lighting or awkward angles.

Initially, I think we can’t rely on high-quality photos, so we need to think of solutions that are robust enough to handle these variations. One idea would be to include clear recommendations in the app on how to take the photos (for example, avoiding direct reflections, ensuring good lighting, etc.), but even so, photo quality could still be a challenge.

It might be worth researching more about image processing techniques that could help mitigate these issues, such as filters to reduce reflections or algorithms that can identify the pieces even under non-ideal conditions. What do you all think?

 

[jroriz]

OT: did you consider ocr on the printed label? There's a suspect #14 repeated a couple of times.. and that appears to be the number of visible mirrors.

The app will be designed specifically to validate the quantity of pieces indicated on the label. That’s why the employee needs to count the pieces manually and then use the app to confirm whether the counted number matches what’s on the label. The idea is to prevent discrepancies and ensure the count is accurate before moving forward with the process. In the case of the repeated number 14, it could be a label error or a coincidence, but the purpose of the app is precisely to help identify and correct this type of inconsistency

 

[udg]

Another OT tip: what if the column of glass plates is positioned on a properly numbered base? Taking the photo for the lower end will display the base so reading the highest number will be easy enough.
A photo showing both the label and the base will give you both quantities to compare.
In alternative to the base, a properly shaped object sporting a visibile scale.
Well, ok. I stop my OT now.
If you asked for image analysis you surely have good reasons.

 

[Sandman]

The app will be designed specifically to validate the quantity of pieces indicated on the label.

I'm curious, how often have they found the number to not match the quantity of pieces? Is this something that happens a couple of times per week, or once every three years, or what?

 

[Daestrum]

Using your image in post #1, I have AI code that can find the panes, but because of the low rez of the image cant find individual panes. Do you have a higher rez image to test with?

 

[jroriz]

The pieces are heavy and difficult to handle. Often, they arrive in bundles of 100 pieces. These bundles are counted before being removed from the containers. This particular bundle in the photo weighs 2 tons.

 

[jroriz]

I'm curious, how often have they found the number to not match the quantity of pieces? Is this something that happens a couple of times per week, or once every three years, or what?

The pieces are expensive, and the quantity is sometimes incorrect. This is not uncommon. Would you like to pay a high price for 20 bananas and only receive 18?

 

[jroriz]

Using your image in post #1, I have AI code that can find the panes, but because of the low rez of the image cant find individual panes. Do you have a higher rez image to test with?

That's exactly what I'm looking for. I'll provide a few more photos, but since I'm traveling, please wait a few more hours.

 

[Sandman]

The pieces are expensive, and the quantity is sometimes incorrect. This is not uncommon. Would you like to pay a high price for 20 bananas and only receive 18?

No, I would not. But If I ordered 100 000 boxes with 20 bananas in each, I wouldn't care as much if one box only had 18.

But I you've absolutely made your case above, they are very heavy and very expensive and are re-packaged - it makes sense to have some sort of help to ensure the correct number of pieces in a bundle. I was just curious.

 

[Daestrum]

This is what it sees in your image (notice the number in the big box). The prompt was simply 'glass panes' The number was how sure it was, not the count.

 

[emexes]

bundles of 100 pieces

Yikes. Any chance of seeing that photo?

 

[jroriz]

This is what it sees in your image (notice the number in the big box). The prompt was simply 'glass panes' The number was how sure it was, not the count.

View attachment 161710

Wow! That was great!
Which API did you use? Google Vision?
Is it paid?
Can you share the code?

 

[Daestrum]

The model I used was "google/owlv2-base-patch16-ensemble" (about 600MB). You would probably need to run it on a PC and call it from Android.