General > General Technical Chat
Is it possible, to have binary (discreet) AI NETWORK with no convolution ?
RJSV:
Hi, I'm building some simple structures to do binary decode, classic EE 101 style, starting with the 4 bits input, then having the explicit output set; 0 through 15.
By analogy, that could resemble the 'expansion' of terms that happens in layers of an AI Network.
Those are all the single connections, to next layer, which I assume have the weights assigned.
With binary, you get a perfect set of permutations or 16 possible states. So, I'm guessing, 'finite state' definition applies...( ??? ).
Then, moving down the signal path, those 16 possible states get interpreted...; meaning that some 4 bit combinations don't make sense, in context. For example, I'm doing Motion Detection hardware, so some combinations of (binary) sensors don't represent any clear movement, left to right.
But, that 'reduction', is being done by design, that being, again as example, motion from rt. to left, or from left to right, forming a 'smaller' set of conclusions.
The very simple set, could conclude; No Action, Motion to Left occurred, or Motion to the right occurred.
Notice that's a REDUCTION, from 16 terms, to more like 3 conclusive terms. So, of course, a Neural Net is going to do this process, between 'Reloe' and 'Pooling',
but I'm curious, how close these two processes are ?
In other words, going from 4 (encoded) bits, to 16 terms, to 3 'summary' terms but without Analog 'weights'... Is this a simplified Network ? Or just some logic custom-wired to a few tasks ? Thanks.
- - Rick B.
rstofer:
Convolution is used for downsizing, typically in image recognition. It is also used for modifying an image to find edges and other features. Blurring is another use for convolution. I don't see where it is applicable to this problem. https://setosa.io/ev/image-kernels/
ReLU, sigmoid and tanh are from a set of possible activation functions and force the output of a layer to constrain values in some range. ReLU eliminates values less than 0 by simply setting the output to 0 when x < 0 or equal to x for x > 0. Sigmoid constrains the values to 0..+1 and tanh constrains the values to -1..+1. You can experiment with this selection but the results will probably be the same regardless of your choice. But it's easy to experiment...
For this problem, the hidden layer(s) will probably use ReLU and the output layer (with 16 outputs) will probably use softmax which will force the outputs to a percentage of probability (the total over all 16 outputs will be 1.0) and the highest probability wins.
https://machinelearningmastery.com/softmax-activation-function-with-python
You can get some ideas from the XOR problem.
https://blog.thoughtram.io/machine-learning/2016/11/02/understanding-XOR-with-keras-and-tensorlow.html
Your problem will have 4 bits of input (the low 4 bits of a 32 bit value) and the hidden and output layers will have 16 outputs. If you only want 3 discrete outputs, make the softmax output layer output 3 values instead of 16. I think the first hidden layer should still have 16 outputs and that might be the only hidden layer. The output layer would just have 3 outputs and still use softmax. Your label data, matching the training/test data, determines what the output will be. It will be a 16 bit representation of the 4 bit input.
You can make training/test data with the numpy randint() function.
If I were doing this, I would use tensorflow and keras. I would get very close to the book "Deep Learning with Python" by Francois Chollet. I like this book a lot. I have several others but this one works for me. And the code just works, no typesetting problems where zero and O or 1 and l get scrambled.
https://www.amazon.com/Learning-Python-Second-Fran%C3%A7ois-Chollet/dp/1617296864
Or I would use the Machine Learning toolbox in MATLAB.
It is very easy to experiment with various models and activation functions when you use something like tensorflow/keras.
I wouldn't use a DNN to solve this problem unless it was an assignment. A simple switch/case statement in C would do the job better.
RJSV:
Thank you, for careful response. (I'm a big fan, of lower level logic, D to A converters, etc.)
I believe the convolution aspect is done manually, in my crude, caracature of a network, that meaning 'staring' at a very few patterns, on notebook there, while deciding which ones imply motion...I'm thinking motion dynamics is very similar to edge detect.
But I'm seeing a similar 'expansion, reduction' cycling, while the data is actually just one bit. That bit originates at the sensor. In the video lecture, Professor Fan discussed a 'real-time Processing Network'. I'm doing 2 bits from sensors, with another 2 bits holding older sample (20 mSec older).
All of this really, for education, on some of the recently popular network theory!
rstofer:
From what little I know, if you want to detect a sequence (sensors changing state), it is a slightly different process than simply decoding the inputs. Here's a paper on the subject: https://arxiv.org/pdf/1802.02046.pdf
I know absolutely nothing about the process but Google turns up hits for 'neural network sequence detection' - no quotes
RJSV:
Thanks, I'm just reasoning things out. Considering having a pair of opto-sensors, 4 meters apart, off on sidewalk. Those could make pulses that you then decode. At 30 meters per second, those two impulses from car motion would be some 100 milli-seconds apart, but also containing info, by which sensor got the (shadow) first.
For example, the classic 'falling edge' for the left side sensor, would be a 'zero', currently, with a 'one' state just recently; That along with a rising edge, on the right side sensor, (similar pattern of older=0, current=1),
that's makes for 2 compelling arguments, to conclude that motion from left to right just occurred.
My point about the 4 bits, 2 of a new input, 1 from each digital sensor, and then another 2 a saved older copy, is that, combinatorially, there are exactly 16 ways that can happen.
Of course, with bigger inputs, that number gets colossal pretty fast. Just with 4 corners, and 2 older copies (shifted), that's 12 bits..., So that's 4096 variations that would need to be checked, if MANUAL pre- determinations are done.
Oh, and also it's the case, where each point relates to 11 others, in space and in the back-time copies, so that there's A LOT of ambiguous patterns, that aren't clearly confused, but don't perfectly indicate some outcome, in this case, some perfectly indicated direction of travel.
Plus, most real-life sensing has the input pulses overlapping, in time.
As for 'POOLING', I've done an approximation, just by dropping half of the sensors, although losing some data. You probably can appreciate, having TWO sensing elements, allows some (crude) estimate, of motion object's speed, at least in terms like:
'Moving slow to left direction'.
I'm also thinking about, using weights after the first stages are digital (digital output from sensors).
Navigation
[0] Message Index
[#] Next page
Go to full version