Machine Learning Algorithms

[BlogRahul]

I don't understand how do we choose  algorithms in Machine Learning. For Example I need  to make model that identify which flower is on the plan t. google search show that we will need CNN algorithm but  I don't understand why the CNN is only useful for this project

[ataradov]

There is no strict logical explanation. They tried a bunch of stuff and the current iteration of the best suitable algorithms for image recognition are CNNs. That is until something else is invented.

Why CNNs are specifically useful for image recognition and classification is pretty easy to see - they iteratively reduce the amount of information down to a small set of numbers.

[tggzzz]

You don't. The "AI" does some magic, you cross your fingers and hope.

"7 Revealing Ways AIs Fail. Neural networks can be disastrously brittle, forgetful, and surprisingly bad at math "
https://spectrum.ieee.org/ai-failures

[RoGeorge]

Different types of NN fits different tasks.  See if these short videos helps
https://www.youtube.com/c/DeepLearningTV/videos
(the title looks like it's about deep learning only, but I remember at some point the series has an overview about which type is good where, I hope I'm not confusing that with some other channel, but even so it's very short and informative as an overview).

Another channel I remember is the Brandon Rohrer's classes/playlists, e.g.:


Also Barry Van Veen has some classes/playlists about AI/ML:
https://www.youtube.com/user/allsignalprocessing/playlists

If these does not link to your exact question, you have to browse through their playlists page and video page of their youtube channels.  See for yourself is any of these suits you.

[SiliconWizard]

ataradov and tggzzz hit the nail.

Current AI with neural networks is essentially based on experiments. The theory behind NNs is relatively simple - at the computation level - as to their ability to "learn", but that's about it. It's almost impossible to predict - and even less so to *prove* how they will perform for a given application.

So people just experiment with some refinements, mainly on the backpropagation steps and learning datasets, AFAIK, compare results and eventually pick the approach which gives the best results overall. To answer your question, people test, test and test more with lots of experiments and data sets until they get good results, then the approach is selected and becomes the de facto one for a given application until the next, better one comes up.

One of the reasons NNs have become useful (while they are certainly nothing new) is that we now have enormous memory and computing power at our disposal for cheap.

And one problem as I said and tggzzz pointed out is that it's impossible to get a formal proof that a given trained NN will behave in the way we expect it to. We can only test, test, test until we get a statistically signficant result that meets our requirements, and it's never 100%. Thing is, what happens for the few % cases in which it fails is unknown (and can be a big risk in any critical application), and *why* it performs as expected is actually also unknown.

Our inability to prove correctness of trained NNs is a major issue, that bites, and will bite us for years to come. Worse yet, analyzing why a trained NN fails for some inputs is also almost impossible. Thus using them in any safety-critical application is a serious problem.

[rstofer]

Google for 'numberphile cnn' (no quotes) and you will turn up a couple of pretty good videos.

Image classification using Python:
https://www.analyticsvidhya.com/blog/2021/06/image-classification-using-convolutional-neural-network-with-python/

MATLAB with the Deep Learning toolbox does a really nice job.  It will also use CUDA units for the math if you have an NVIDIA graphics card and the Parallel Programming toolbox.  Always another toolbox...

There are a lot of books on the topic.  I have a bunch of them but it's still slow going.

[tggzzz]

And one problem as I said and tggzzz pointed out is that it's impossible to get a formal proof that a given trained NN will behave in the way we expect it to. We can only test, test, test until we get a statistically signficant result that meets our requirements, and it's never 100%. Thing is, what happens for the few % cases in which it fails is unknown (and can be a big risk in any critical application), and *why* it performs as expected is actually also unknown.

Our inability to prove correctness of trained NNs is a major issue, that bites, and will bite us for years to come. Worse yet, analyzing why a trained NN fails for some inputs is also almost impossible. Thus using them in any safety-critical application is a serious problem.

It is even worse than that You have no idea of how close you are to the boundary where they stop working. There are many examples of making trivial (even invisible) changes to pictures, and the classifier completely misclassifies the image.

[RoGeorge]

Same things can be said about humans.  But this wouldn't answer the question from the OP.  There are places where ML fits and other places where it doesn't.

My advice is to follow the classes then go and experiment, and it will all start to make sense.

[tggzzz]

Same things can be said about humans.  But this wouldn't answer the question from the OP.  There are places where ML fits and other places where it doesn't.

Key differences: the human can

• explain why they made a decision. That's an unsolved problem for current ML programs
• can get them to do differently next time. You can't train an ML program that easily, since you don't know
  
  • why it made a decision
  • what extra examples are necessary
  • how those extra examples would have changes previous and future decisions
  so ML is being and will be used inappropriately as "the computer says so" magic

Those problems are critical, and peoples lives have been and are being destroyed by them. There are many examples reported in comp.risks. I'll just note that

• some US states are using ML to define whether unconvicted crime suspects should be held in jail, and the sentence of convicts
• some medical diagnostic systems have been found making decisions on the basis of the font on X-rays

[ralphrmartin]

I strongly recommend
Deep Learning with Python, Second Edition

https://www.manning.com/books/deep-learning-with-python-second-edition

which not only shows how to apply deep learning using tensorflow, but also explains the ideas behind it. Your question and many others will be answered by this book.

[diyaudio]

I don't understand how do we choose  algorithms in Machine Learning. For Example I need  to make model that identify which flower is on the plan t. google search show that we will need CNN algorithm but  I don't understand why the CNN is only useful for this project

Interesting question, radically two different domains.

- Machine Learning (Algorithm) vs CNN (Deep Learning Network)
- Image detection using "Computer Vision algorithm" vs Image detection using "Vision Deep Learning Network"

Machine Learning - high level algorithms for statistical related predictions. 

Examples:

- Density-based techniques (k-nearest neighbor ocal outlier factor, isolation forests.
- Subspace, correlation-based and tensor-based outlier detection for high-dimensional data.
- One-class support vector machines.
- Replicator neural networks., autoencoders, variational autoencoders, long short-term memory neural networks.
- Bayesian networks.
- Hidden Markov models.
- Minimum Covariance Determinant.
- Cluster analysis-based outlier detection.
- Deviations from association rules and frequent itemsets.
- Fuzzy logic-based outlier detection.
- Ensemble techniques, using feature bagging.

DNN (Deep Learning Network)

Used to find "complex relationships" in "complex domains" (not limited to) Image/Video/Audio/Speech ect.. DNN has a high degree of accuracy, they do however require large datasets to build models that performs a prediction. Note: we no longer dealing with "algorithms here" but "network configurations".. algorithms has a predicted outcome for a given input,  DNN doesn't, there is no way to reverse engineer how the network came to a conclusion as we can see anything from a few 10, 30.. 100k neurons connections internally to a million or so dealing with very "abstract data points/patterns" modeling how the brain work when we make predictions about sound or image objects. (amazing work this last 10 years and its exponentially getting better as the research progresses)

Examples:

Image Detection Networks:

- YOLO https://en.wikipedia.org/wiki/Object_detection
- ResNets https://towardsdatascience.com/cnn-resnets-a-more-liberal-understanding-a0143a3ddac9
- Inception network https://medium.com/ml-cheat-sheet/deep-dive-into-the-google-inception-network-architecture-960f65272314

Audio Speech Detection Networks:

https://www.analyticsvidhya.com/blog/2018/01/10-audio-processing-projects-applications/

- Audio Classification
- Audio Fingerprinting
- Automatic Music Tagging
- Audio Segmentation
- Audio Source Separation
- Beat Tracking
- Music Recommendation

[rstofer]

So you want to identify dogs and you have thousands of photos of various breeds.  Unfortunately, all of the photos of one particular breed show the dog on a couch in various poses.  The machine will learn to identify the couch as a dog and completely overlook the dog.  Every couch looks like a dog of a specific breed!

ML is hard!  The matrix math and the gradient descent is easy to understand (just a LOT of partial derivatives) but true knowledge isn't easy.  Teslas still have a tendency to run into emergency vehicles - police cars and fire trucks.  Waymo robotaxis don't do well with traffic cones either.

https://www.theverge.com/2021/9/14/22673497/tesla-nhtsa-autopilot-investigation-data-ford-gm-vw-toyota
https://www.zdnet.com/article/waymo-robotaxis-struggle-to-appropriately-react-when-around-traffic-cones

Getting an identification is easy.  Getting the RIGHT identification is a good deal harder.

[SiliconWizard]

Yes. As a thought, I find it interesting how "AI" has become so popular while being, in a way, the antithesis of what computer science is usually about - for instance, proving correctness, defining and analysing algorithms, etc.

We should not confuse different levels of concepts here: while the algorithms used for ML and NNs are well understood, and relatively simple, they, by themselves, do not do anything "useful" regarding the task we're interested in - for instance, image recognition. The "networks" they create is what does the interesting part, and so they kind of implement ""hidden" algorithms that we are unable to analyze. Since we can't really analyze them, is this still really science? tggzzz said it's a bit of "magic", and in a way, that's exactly it as far as we are concerned.

Just because we have formalized ways to "induce" this magic doesn't really make it less magic.

Another thought is that it's, at first, all based on trying to mimick neural-based living intelligence. Thing is, we have captured only a part of what it is. Maybe just a small part. It's not just about the amount of neurons and learning data, IMO. Neural structures, AFAIK, in living beings do not just merely form based on inputs. We know there are certain structures that form, almost always in the same way and at the same locations (in the brain for instance), "coding" specific things with a specific level of abstractions. That part of "intelligence" mostly eludes us in AI at the moment, and that's only just scratching the surface.

[tggzzz]

So you want to identify dogs and you have thousands of photos of various breeds.  Unfortunately, all of the photos of one particular breed show the dog on a couch in various poses.  The machine will learn to identify the couch as a dog and completely overlook the dog.  Every couch looks like a dog of a specific breed!

And then one photo contains a dog on a couch that is 5mm higher than other couches. The machine classifies that as a table, and so identifies the dog as a different breed. But the second photo in that series is from a slightly different angle so the height is misestimated and the original "correct" breed is diagnosed

[diyaudio]

Yes. As a thought, I find it interesting how "AI" has become so popular while being, in a way, the antithesis of what computer science is usually about - for instance, proving correctness, defining and analysing algorithms, etc.

We should not confuse different levels of concepts here: while the algorithms used for ML and NNs are well understood, and relatively simple, they, by themselves, do not do anything "useful" regarding the task we're interested in - for instance, image recognition. The "networks" they create is what does the interesting part, and so they kind of implement ""hidden" algorithms that we are unable to analyze. Since we can't really analyze them, is this still really science? tggzzz said it's a bit of "magic", and in a way, that's exactly it as far as we are concerned.

Just because we have formalized ways to "induce" this magic doesn't really make it less magic.

Another thought is that it's, at first, all based on trying to mimick neural-based living intelligence. Thing is, we have captured only a part of what it is. Maybe just a small part. It's not just about the amount of neurons and learning data, IMO. Neural structures, AFAIK, in living beings do not just merely form based on inputs. We know there are certain structures that form, almost always in the same way and at the same locations (in the brain for instance), "coding" specific things with a specific level of abstractions. That part of "intelligence" mostly eludes us in AI at the moment, and that's only just scratching the surface.

oh don't sound so hateful, you telling your age.     
 

[rstofer]

In this video, Andrew Glassner talks about the dangers of deploying AI and using machines to make decisions about jobs, home purchases, duration of incarceration (that may be in a different video) and so on.

See around 16:00



In a Glassner video I was watching yesterday (don't know which one), he mentioned an AI that was used to determine the likelihood of recidivism and recommend sentencing.  The defense was unhappy with the outcome and want to interrogate the 'expert witness' (the machine).  The manufacturer said "No!" and that's where it died.  Nobody is accountable for the sentencing guidelines and, of course, they tend to be on the long side.  See around 19:30

It's a slippery slope when you can't even see the model much less the internals.

[SiliconWizard]

Yep indeed. With the unability to explain or analyze decisions made with AI comes the question of accountability. People will be happy to reap the benefits of AI, but nobody will ever want to be held accountable for any mishap. Yes, a bit like politics.

[Siwastaja]

Also lack of context understanding.

I like to refer to an example where a supposedly state-of-art image recognition "AI" recognizes objects on a street, draw the bounding box in real time and puts the label next to it. The typical demo.

But then, the rectangular thing attached on the outside of a house at the end of the driveway, which we humans call "garage door", is misidentified as "flat screen TV", and indeed, if you just cropped that part of an image, a human could make the same mistake - it's just a rectangle with little or no details. What makes it a garage door, is the context around it. You don't buy a 300-inch flat screen TV, and you don't mount it outside your house, on ground level, at the end of your driveway. This is all obvious to a human.

[ralphrmartin]

But the second photo in that series is from a slightly different angle so the height is misestimated and the original "correct" breed is diagnosed

It's standard in these approaches to augment the original training data by adding copies of it in many different orientations and at many different scalings, to try to overcome issues like this.

Nevertheless, a major problem is having sufficient training data which covers all the bases. A self driving car trained only with data from California is unlikely to do well on the very different roads in Wales, for example.

You also have to be very careful about how many false positives and false negatives are acceptable. Facial recognition systems are clearly a case in point.

[ralphrmartin]

With the unability to explain or analyze decisions made with AI ...

The situation isn't as bad as you are making out. For example, if you have a look at Chollet's book I mentioned earlier in this thread, you will see there are ways to include visualizations of the data in intermediate layers in deep learning systems, to show e.g. which areas of an image the system is using to make its classifications.

[tggzzz]

But the second photo in that series is from a slightly different angle so the height is misestimated and the original "correct" breed is diagnosed

It's standard in these approaches to augment the original training data by adding copies of it in many different orientations and at many different scalings, to try to overcome issues like this.

How many near copies are needed? One? Ten? Hundred? In a finite-sized machine, which other training examples do you discard?

Would it recognise a person with one leg and a crutch as a person?

Nevertheless, a major problem is having sufficient training data which covers all the bases. A self driving car trained only with data from California is unlikely to do well on the very different roads in Wales, for example.

How can you tell when you have sufficient training data?

If not California, then what would happen in Oregon or British Columbia? Would it be OK in London or Rome? Where are the boundaries?

In a finite-sized machine, which other training examples do you discard?

[SiliconWizard]

With the unability to explain or analyze decisions made with AI ...

The situation isn't as bad as you are making out. For example, if you have a look at Chollet's book I mentioned earlier in this thread, you will see there are ways to include visualizations of the data in intermediate layers in deep learning systems, to show e.g. which areas of an image the system is using to make its classifications.

You're right, but that's still very, very far from the point where you can actually fully determine what happens in particular cases and make sense of that.
There is, and sure will be continuing research in that particular area of neural network analysis, because it's obviously a major concern. No doubt things are going to improve.
But analyzing neural networks is inherently a NP-complete problem. Those are a bitch to tackle. The training part itself is already NP-complete.
https://www.sciencedirect.com/science/article/abs/pii/S0893608005800103

Point here is, it's not about how you can partially analyze parts of the neural network - which is certainly interesting per se - but being able for sure to determine how and why a certain network came to a given conclusion for a given set of inputs, in order to 1/ know if we can trust the result - and thus should apply it, and 2/ determine accountability. It doesn't matter much for any non-critical task of course. I don't mind if Google image has occasionally classified a bird as a motorbike. But for any critical one, certainly does.

[ralphrmartin]

... being able for sure to determine how and why a certain network came to a given conclusion for a given set of inputs, in order to 1/ know if we can trust the result - and thus should apply it, and 2/ determine accountability.

You are thinking about these systems as if they should be deterministic - like most software to date. Yet human beings don't work like that. You would never get that certainty from a human, and if they are able to justify why they did something, it is usually a post-hoc explanation cooked up as a justification. You might be surprised to see how much experts disagree in interpreting x-ray images  when trying to spot cancer, for example.

Don't get me wrong. Some software should be deterministic - particularly safety critical systems - where we understand the requirements exactly. But in other cases, the answer does not have to be perfect. The system just has to produce the right answer more often than a human expert - and the people using it need to realise its limitations.

Indeed, for dull repetitive tasks (like monitoring a live video stream for the presence of one or more persons), it may not take much to beat human performance, as people get distracted, bored, etc.

Going back to "how much training data do we use?", and also asking "how do we know how well it works?", a key part of the answer is to use separate validation sets from the training data, to measure generalizability to new data.

And as for discarding data for a finite sized machine - the simple answer is that you input the data in batches. You don't try to input it all at once. The issue is collecting enough data for good generalizability, rather than too much data.

[tggzzz]

Determinism is not required. There are many useful nondeterministic systems.

Predictability is required.

In order to be predictable, the designer and/or reviewer must understand what is happening and why.

The designers do not - and unless there are fundamental advances - cannot understand why a result appears, nor of duct what the result will be. They can hope, but that isn't sufficient.

For example, if a judge incarcerates/frees you, they are expected to explain why that is the appropriate result. When an ML machine does the same, there is no explanation. Yes, that is already happening in the USA.

Ditto a car running down a pedestrian or driving into a roadside barrier.

Fundamentally the central point about science is being able to male correct predictions. Magic and cargo-cult science aren't constrained by that.

[snarkysparky]

Some situations where being right on average are good setups for AI.
Like sorting recyclables on a conveyor belt.  The missed edge cases are no more costly than a missed central case.

Like the casino.  They only need win 51% of the time.

But it seems AI is being applied to exclusively situations where this constraint is not acceptable.   Replacing human judgement in decisions that directly impact other humans for instance.   We don't want to accept a wrong decision as " oh well sht happens "

I am in the very skeptical camp about it.  I don't think any system will ever get situational awareness in a driving situation beating a functional human in the edge cases.
I don't want a self driving car to make a mistake and run into me and then be told self driving cars are better than the average human.   I expect the human even if below average to take responsibility for his actions.  I know...  frequently doesn't happen.