Recommendation for Machine Learning book

[davorin]

Good afternoon (o;

As I want to start with ML, especially with audio detection, I would like to ask if someone can recommend an ebook (or more) for a complete starter in ML.


thanks in advance
davorin

[Faranight]

Audio detection is basically picture recognition after processing the audio clip into a log-mel FFT diagram.

My personal recommendation:
https://www.oreilly.com/library/view/hands-on-machine-learning/9781098125967/

[davorin]

That one I just looked up few minutes ago (o;

But is that understandable for a novice like me?
Well..doing electronics, software and python since many years though...but algebra is not my field (o;

[Terry Bites]

AI is a complex multifaceted subject. I don't imagine there would be a novice level entry point unless you're already an experienced computer scientist.

https://web.stanford.edu/~jurafsky/slp3/
https://towardsdatascience.com/how-amazon-alexa-works-your-guide-to-natural-language-processing-ai-7506004709d3

[davorin]

Hmmm…I thought that ML and AI would be different topics…only somehow related…

[KE5FX]

What people call "ML" and "AI" has converged in the last few years under the term "Deep Learning," named for what has become everybody's favorite tool (a perceptron or convolutional NN with more than two layers.) 

If you are already a Python person then you're ahead of the game, because that's where 99% of the work is done nowadays.  Likewise, if you don't have a good background in both linear algebra and statistics... well, you will, but it will cost you a lot of additional learning time.

Karpathy's "Neural Networks: Zero to Hero" playlist on YouTube is a good way to get started.  A good exercise is to follow along and implement the same rudimentary language model in a different programming language. 

[davorin]

Thanks for the heads up and the pointer ;-)

Is there a big benefit going for a nice Cuda GPU or is an old 6 years old Debian PC good enough to get started with training models?

What I have in mind is to start with the Nicla Voice platform from Arduino. Dunno how well it ffits in the PyTorch flow as many companies now offer online model training tools where it can get pretty expensive (o;

First application is to detect with audio if a welding process has gone good or bad…pretty strange…I know ;-)

[KE5FX]

Thanks for the heads up and the pointer ;-)

Is there a big benefit going for a nice Cuda GPU or is an old 6 years old Debian PC good enough to get started with training models?

What I have in mind is to start with the Nicla Voice platform from Arduino. Dunno how well it ffits in the PyTorch flow as many companies now offer online model training tools where it can get pretty expensive (o;

First application is to detect with audio if a welding process has gone good or bad…pretty strange…I know ;-)

If you start with the basic nuts and bolts, as in the Karpathy videos, you don't need a GPU at all.  Once you get farther, yes, you'll want GPUs (note plural), but you probably cannot afford to buy the hardware you want, so you'll probably end up renting it from a cloud provider. 

Basically, whether you are talking about R&D work or learning, there is a relatively narrow time segment where owning a serious GPU rig makes sense.  It is probably better to get comfortable with the cloud options sooner rather than later, but others may have different opinions there.

[davorin]

Well our welding detection project should show some results by end of this year…so no rush here (o;

So plenty of time to start learning…maybe other projects or ideas would benefit.

Basically I‘m just curious if I could master this field with 56 as I started in a much younger age with vacuum tubes and 20mm soldering pitch ;-)

My boss already allowed me to buy some ebooks and ML boards to get started…

[tszaboo]

About 90% of the work in ML is data collection and classification by hand. The rest is the programming.
You don't need CUDA or anything else hardware wise to train audio models. Your data model is probably going to be 10000x smaller than image classification or generation models, it's going to go fast. Unless you want a PC anyway.
Learn python.
Forget the boards for now, make it work on a PC. No, seriously, all that is just noise for starters, you need to figure out much bigger things than what hardware to run it on.
Also, you should make a sanity check if you need ML/AI at all for it. I had a project that I sunk a lot of time trying to force in Pytorch machine learning, only to solve it with scipy in fraction of the time with better result.

[rstofer]

Here is the MATLAB code for the famous Digit Recognition problem.  Training takes 13 seconds on a Macbook Pro M2 and yields 99.64% accuracy which is pretty spectacular.  Total elapsed time is around 30 seconds to read and set up the input data.

Code: [Select]

%Program to recognize digits using Deep CNN

% Console commands to download 1dataset
% >> dataFolder = fullfile(toolboxdir('nnet'),'nndemos','nndatasets','DigitDataset');
% >> imds = imageDatastore(dataFolder, ...
%    'IncludeSubfolders',true,'LabelSource','foldernames');
%
% Dataset is extracted to:
% C:\Program Files\MATLAB\R2022b\toolbox\nnet\nndemos\nndatasets\DigitDataset

tic % start the clock

%Giving path of dataset folder
digitDatasetPath='/Users/rstofer/Digits/'; % was c:\Digits for Windows

%Reading Digit Images from image Database Folder
digitimages=imageDatastore(digitDatasetPath,'IncludeSubfolders',true,'LabelSource','foldernames');

%Distributing images in the set of Training and Testing
%There are 1000 images of each digit
numTrainFiles=750; %numTrainFiles=-.75  (75%)
[TrainImages,TestImages]=splitEachLabel(digitimages,numTrainFiles,'randomize');

layers=[
imageInputLayer([28,28,1],'Name','Input')

    convolution2dLayer(3,8,'Padding','same','Name','Conv_1')
batchNormalizationLayer('Name','BN_1')
reluLayer('Name','Relu_1')
maxPooling2dLayer(2,'Stride',2,'Name','MaxPool_1')

convolution2dLayer(3,16,'Padding','same','Name','Conv_2')
batchNormalizationLayer('Name','BN_2')
reluLayer('Name','Relu_2')
maxPooling2dLayer(2,'Stride',2,'Name','MaxPool_2')

convolution2dLayer(3,32,'Padding','same','Name','Conv_3')
batchNormalizationLayer('Name','BN_3')
reluLayer('Name','Relu_3')
maxPooling2dLayer(2,'Stride',2,'Name','MaxPool_3')

convolution2dLayer(3,64,'Padding','same','Name','Conv_4')
batchNormalizationLayer('Name','BN_4')
reluLayer('Name','Relu_4')

fullyConnectedLayer(10,'Name','FullyConnected')
softmaxLayer('Name','SoftMax')
classificationLayer('Name','OutputClassification')
];

lgraph = layerGraph(layers);
plot(lgraph); %Plotting Network Structure

%------ Training Options ------
options = trainingOptions('sgdm',                        ...
                          'Shuffle','every-epoch',       ...
                          'ExecutionEnvironment','auto', ...  % or AUTO, GPU or CPU
                          'InitialLearnRate',0.01,       ...
                          'MaxEpochs',4,                 ...
                          'ValidationData',TestImages,   ...
                          'ValidationFrequency',30,      ...
                          'Verbose',false,               ...
                          'Plots','training-progress');

net = trainNetwork(TrainImages,layers,options); %Network Training
% analyzeNetwork(net)
YPred = classify(net,TestImages); %Recognizing digits
YValidation = TestImages.Labels; %Getting labels'
accuracy = sum(YPred == YValidation)/numel(YValidation); %finding accuracy
fprintf('Accuracy: %g%%\n',100*accuracy)
toc


The Pytorch code is quite similar and the toolchain is free.
https://towardsdatascience.com/handwritten-digit-mnist-pytorch-977b5338e627