EEVblog Electronics Community Forum
Electronics => Projects, Designs, and Technical Stuff => Topic started by: mikeselectricstuff on April 27, 2014, 09:28:56 am
-
Some thoughts and design details on how you might make a relatively low-cost camera array for interesting video effects
Designing a low-cost "bullet-time" camera rig (https://www.youtube.com/watch?v=Lvyslu65V3s#ws)
-
Really nice and useful video! :-+ :-+ :-+
Alexander.
-
I think there is shielded cat-5 connectors that could solve the leading ground issue.
Any other suitable connector to use with cat-5 cable out there?
Alexander.
-
Nice video Mike, even with the wondering in the middle if anybody was still awake at that point.
Very nice idea there, and a nice set of implementation and possible issues with each and the advantages and implications of them all. No idea if i would ever use one, but the idea of having a massive wall of the small camera and displays and making an active mirror seems like a very nice idea. Might be very interesting at a gallery as an entrance display if it was about 5m long and at around eye height.
-
For background information, Mike is talking about this rig:
http://youtu.be/hgjAKzzDQP8 (http://youtu.be/hgjAKzzDQP8)
It would be very nice to watch shots of Mike's Destruct-O-Tron or a Jacob's ladder with such a rig.
I thought once, getting many of World’s Shittiest Camcorder: Teardown in EEVblog #595 and hack them to sync together. Only $7.5 per unit. If you can trigger the event you want to shot at the exact moment of those cameras shooting together, the rest will be easy in PC.
I would rather prefer to watch a custom FPGA synchronized solution based on Mike's original plan :-+.
-
Great video Mike !
You asked if anyone wanted to build this rig and I'd love to :) ! I'm a student in France and I would like this to be my team project for next year. Would that be ok with you ? Or is it stealing :P. I think that with a team of engineers on this we could build a very complete product/prototype with a nice enclosure, good software and good hardware.
I have experience with FPGA's and microcontrollers but I have never used them with cameras. I also have no experience with this sort of big installation with many modules and a control box connected to a PC. I'll start looking for material about these aspects online.
-
For background information, Mike is talking about this rig:
http://youtu.be/hgjAKzzDQP8 (http://youtu.be/hgjAKzzDQP8)
It's kind of ironic that this is such a bad video.
-
I used to work for Video Production industry in those days ;) . You can see that Behind The Scenes are SD Video. Most likely shot on (mini)DV, Betacam, 8mm and even VHS. There were no any mini DVD Disc or Hard Disk video camcorder.
Some more interesting Behind The Scenes videos for better explanation of the technique:
http://youtu.be/JjPBsx_eWs4?t=2m30s (http://youtu.be/JjPBsx_eWs4?t=2m30s)
http://youtu.be/Kjcv-JtUOgA?t=38s (http://youtu.be/Kjcv-JtUOgA?t=38s)
http://youtu.be/2f79nuDTcCA?t=39s (http://youtu.be/2f79nuDTcCA?t=39s)
The 1st and Last cameras were normal 35mm Film Cinema cameras. If memory serves me well, the other cameras were Nikon D1 (1st Digital Nikon DSLR).
Nowadays, it is easy to rent such a rig for cheap commercials.
-
Great video Mike !
You asked if anyone wanted to build this rig and I'd love to :) ! I'm a student in France and I would like this to be my team project for next year. Would that be ok with you ? Or is it stealing :P. I think that with a team of engineers on this we could build a very complete product/prototype with a nice enclosure, good software and good hardware.
I have experience with FPGA's and microcontrollers but I have never used them with cameras. I also have no experience with this sort of big installation with many modules and a control box connected to a PC. I'll start looking for material about these aspects online.
Go for it !
-
I liked your idea on the setting up the individual IDs on the camera modules. I like the idea also of an "infinitely" expandable series by just adding more and more to it.
My thoughts on snapshot latency would be maybe a dedicated line that all the modules are tapped into. When the control module wants all the cameras to fire, it pulls the line high to trigger a snap event.
Then the camera would dump the data into the memory and hold. If the memory was big enough, after the snap takes place, the FPGA could process the data and store the compressed version of the picture in another part of RAM.
If the FPGA has an Ethernet MAC address set, that could be the self identifier. When the modules power up they could take 2 or 3 seconds and broadcast their MAC address to the left and right. Then the modules would know who was there and a sort of routing table or address resolution table could be made.
The reason I thought of that was it might make it easier for these devices to self configure in a grid configuration because you'd have each module surrounded by 4 others.
It might even make it possible to expand these while live. Modules would identify themselves on startup.
As I had mentioned in the youtube video comments, you could have the cameras self correct for small alignment issues by taking more of a picture than was needed and then trim off the excess. If you had a single source of alignment in the middle the cameras could all take a picture and send it back. Then the computer could decide how much of a shift would be required on each picture. Then without touching the cameras, place the object in the middle to be photographed.
-
My thoughts on snapshot latency would be maybe a dedicated line that all the modules are tapped into. When the control module wants all the cameras to fire, it pulls the line high to trigger a snap event.
No need - the forward data channel should easily be fast enough to send a trigger signal. if there is cam-to-cam latency due to data decode/reencode, this could be compensated for, at the cost of a slight delay, but certainly not enough to be worth adding extra cabling. If you really wanted to you could add an out-of-band signalling mode to the protocol to allow a fast trigger signal to bypass any encode/decode - e.g. a UART break character, or an illegal bit period on a manchester code.
Then the camera would dump the data into the memory and hold. If the memory was big enough, after the snap takes place, the FPGA could process the data and store the compressed version of the picture in another part of RAM.
Perhaps, but I'm not sure how much useful compression you could do with a low-end FPGA
If the FPGA has an Ethernet MAC address set, that could be the self identifier. When the modules power up they could take 2 or 3 seconds and broadcast their MAC address to the left and right. Then the modules would know who was there and a sort of routing table or address resolution table could be made.
Some FPGAs, and some SPI flash chips have a unique ID.
If all nodes have an ID, an alternative addressing scheme could use that, along with a discovery process, where the host simply says "send me your ID". The first time, the first cam in the chain would send, the replies from the downstream ones would be ignored as the first cam would have control of the return path. Once you know the ID of the first, you tell it via its ID to ignore any future send-ID commands. You then repeat this process to discover the ID of each cam in turn.
However I'm not sure this would have any advantage over my proposed method, unless there was a reason you'd want to keep track of physical addresses of specific cameras regardless of connection order - maybe to have per-cam calibration factors.
The reason I thought of that was it might make it easier for these devices to self configure in a grid configuration because you'd have each module surrounded by 4 others.
I don't see any point in anything more than a linear chain, and many downsides - if you want a grid, just zig-zag the line.
and these while live. Modules would identify themselves on startup.
-
Some thoughts and design details on how you might make a relatively low-cost camera array for interesting video effects
You made it sound so simple! Thanks for sharing your thought process. I'd be starting from ground zero on this, but would like to have a go at even getting one camera up and running. I don't recall a video of you making the combined screen and camera you showed at the start. Any recommendations from your research on a basic camera module that has available connectors and a straightforward interface?
Anthony
-
That was a great video Mike, I totally enjoyed that.
-
Great video Mike!
I was awake through the whole thing! ^-^
Love that address assignment scheme! Would that not have worked on that MC array where you had to manually pull an external pin low that you talked about on the AmpHour?
-
Some thoughts and design details on how you might make a relatively low-cost camera array for interesting video effects
You made it sound so simple! Thanks for sharing your thought process. I'd be starting from ground zero on this, but would like to have a go at even getting one camera up and running. I don't recall a video of you making the combined screen and camera you showed at the start. Any recommendations from your research on a basic camera module that has available connectors and a straightforward interface?
Anthony
There are quite a few - OV7670, OV9655, OV9650 and OV2640 are the most commonly available cheap modules but there are others.
Available datasheets are a bit sketchy, but sets of register values can usually be found, e.g. in Linux drivers and devboard examples, and once you have something coming out you can then experiment with tweaking values.
All the cheap modules use a standard 0.5mm 24 way FFC connector, and the pinout is reasonably standard ("Sunny" OV9655 modules being the exception), but if necessary, they are cheap enough to sacrifice a module & unsolder the chip & meter out the PCB
The data interface is standard and very simple - 8 or 10 data bits, a pixel clock, plus horizontal & vertical sync.
Earlier camera video :
Nanocam (https://www.youtube.com/watch?v=CsKd8wNWzYU#ws)
-
Great video Mike!
I was awake through the whole thing! ^-^
Love that address assignment scheme! Would that not have worked on that MC array where you had to manually pull an external pin low that you talked about on the AmpHour?
No, because that had all devices connected in parallel, not in a chain.
-
Great video Mike!
I was awake through the whole thing! ^-^
Love that address assignment scheme! Would that not have worked on that MC array where you had to manually pull an external pin low that you talked about on the AmpHour?
Which AH episode was Mike on?
http://www.theamphour.com/the-amp-hour-135-x-ray-examining-xenogogue/ (http://www.theamphour.com/the-amp-hour-135-x-ray-examining-xenogogue/)
-
Mike,
I saw a video recently of a group of individuals that did "Bullet Time" with 50 GoPro Black Edition cameras. It was pretty cool but very costly for 50 GoPro cameras. I'll try to find the video and link it in here.
-
On the final shot is that biggest camera from bottom an infrared camera? It has that kind of lens which looks like infrared camera.
If it is infrared, any videos coming out?
Raspberry pi bullet time:
Bullet Time Style Effect with PiFace Control and Display and Raspberry Pi (https://www.youtube.com/watch?v=IqoA4HeBCQ4#ws)
http://makezine.com/2014/02/28/building-a-bullet-time-rig-at-the-raspberry-pi-jamboree/ (http://makezine.com/2014/02/28/building-a-bullet-time-rig-at-the-raspberry-pi-jamboree/)
Anyways thanks for very informational video, I hope you make more in the future. :-+
-
Alternative solution?:
Are there any camera modules which have an output inhibit or sleep mode?
You could connect camera modules onto one bus (using an LVDS switch) and then sequentially capture one image from each?
That way, each board can be fairly dumb: it can have 8 camera connectors, and a shift register on it for the enable/control.
Store the images in a large-ish frame buffer (~512MB-1GB)
What would kill this is if the camera modules take a long time to initialise after leaving sleep.
So an alternative is to find some way to get them on all the same Vsync (perhaps a common clock) and switch LVDS on each Vsync only, thus it looks to the master, that it is only one camera.
You could use a PIC12F to handle I2C interface, etc. for each camera and maybe some camera modules will output a Vsync signal.
You could also try with PAL cameras if picture quality is not critical as Vsync is readily available from any LM1881 or LMH1980 chip.
-
Maybe "World's Worst Shittiest Camcorder" according to Davecan be used
http://youtu.be/1EuBRmfJLSk?t=3m50s (http://youtu.be/1EuBRmfJLSk?t=3m50s)
The sensor is already nicely mounted, and cost less then buy component from digikey ;D
-
You can go on eBay and pawn shops and search for lots of Canon PowerShot cameras that work (in the sense that the lens work).. lots of these show up because the battery compartment plastic lid breaks and batteries no longer stay inside.
A lot of these older models are supported by CHDK (google it), a custom firmware that can be loaded from SD card and doesn't overwrite the existing firmware (so it's relatively safe) and enables lots of features like snapping pictures on remote control press, or when infrared led is detected, timelapse, motion detection etc etc ... hell, if the camera is powerful enough you can even play games on it, or do LUA scripting with the functions.
Consiering the quality increase (can't compare a canon 4-8 megapixel with that vga sensor with plastic lens) ... it's somewhat of a nobrainer.
edit: ah yes, it's been done as i described above, with chdk and 10 cameras : http://www.geekosystem.com/tesla-coils-bullet-time/ (http://www.geekosystem.com/tesla-coils-bullet-time/)
reson8 on Vimeo (http://vimeo.com/24977187)
-
What about have the command/control line (UART or similar) be a single line pass through on each module, with the high speed return pair chained as described? You could initialize it by sending a command to put all modules into a mode where they all send a predefined bit pattern. Then the module that doesn't receive that bit pattern is the last one on the chain and assigns itself address 0, then tell the next module to name itself 1, that tells the next to name itself 2, etc.
-
At some in the video, Mike wondered whether it might be possible with JPEG-enabled camera modules to retrieve data at a more reasonable rate. For what it is worth, I have a data point on that.
A number of years ago, I did a design around a cell phone camera module that had integrated JPEG.
I did try this approach (slowing the data). It did work, but the results were not appealing.
The JPEG compression was accomplished by the module in real-time as the data is being read from the CMOS imager. Only sixteen (height of the JPEG macroblock) horizontal lines worth of uncompressed image need to be maintained. (*1)
As new raw image data got shifted in, compressed bits would intermittently spill out. So, although the overall output data rate was quite small, it happened in momentary bursts at the same rate as would the raw uncompressed data.
The only way to get the data rate at a manageable point was to slow the frame rate right down (say, 5 Hz or less). At that point, the rolling shutter effect was particularly annoying.
*1: Given that a smaller amount of RAM is required to do JPEG compression on the fly (since one doesn't need RAM to store an entire raw image) and there are an increasing number of 100+ MHz ARM processors with camera interfaces and a modest amount of internal SRAM, I wouldn't completely discount the notion of a single chip microcontroller (without SDRAM and in lieu of the PLD and SPI Flash).
I really liked Mike's notion of monitoring the SYNC signal from the module and gapping the clock to achieve system-wide synchronization. My experience was that these modules had a mind of their own in response to I2C commands and latency wasn't particularly consistent. It was very annoying.
My only reservation is that some of these modules are using a PLL. If they do, they might either behave badly or be oblivious to the gap.
I've also thought before about this "bullet time" design problem. In addition to the CHDK approach already mentioned, I noticed that there are USB camera boards available. It is only one data point, but there is an Indian company (search for "e-con") that has some products. There is a 2Mpixel module for $19/ea in 1k quantity, although there is no indication as to synchronization trigger. A more expensive 5Mpixel module for $35 in 5k quantity specifically mentions an external hardware trigger. Perhaps it is not as fun to buy it off the shelf, but I decided that the amortized engineering effort for a custom solution was going to cost me more if I decided to do it.
As far as inexpensively (or at least, less expensively) mounting the cameras, the form factor of the e-con modules made me think of using small diameter PVC pipe. It could stand vertically with a base consisting of a block of wood with a drilled hole for the PVC pipe.
-
I've been having some success using an Arduino with an Altera CPLD and a 4Mbit sram chip to buffer the data from an OV2640 board. The problem is I cant get the config for the camera board to produce the 640x480 JPEG image I want. I did see a page on a chinese web site (lost the link but I'll find it & stick it on here or www.ianrolfe.com (http://www.ianrolfe.com) when) where someone has got it to work but that appears to be the only place :)
My next move it to dissect the linux driver (which doesn't seem to support JPEG) & see if I can combine it with rhe google translate of the website to get something working!
Ian
-
1 trillion frames??!?!? :wtf:
http://www.iflscience.com/technology/laser-pulse-imaged-through-bottle-1-trillion-frames-second (http://www.iflscience.com/technology/laser-pulse-imaged-through-bottle-1-trillion-frames-second)
-
I've been having some success using an Arduino with an Altera CPLD and a 4Mbit sram chip to buffer the data from an OV2640 board. The problem is I cant get the config for the camera board to produce the 640x480 JPEG image I want. I did see a page on a chinese web site (lost the link but I'll find it & stick it on here or www.ianrolfe.com (http://www.ianrolfe.com) when) where someone has got it to work but that appears to be the only place :)
My next move it to dissect the linux driver (which doesn't seem to support JPEG) & see if I can combine it with rhe google translate of the website to get something working!
Ian
These people appear to have examples for JPEG from the 2640
http://www.arducam.com/ (http://www.arducam.com/)
-
I've been having some success using an Arduino with an Altera CPLD and a 4Mbit sram chip to buffer the data from an OV2640 board. The problem is I cant get the config for the camera board to produce the 640x480 JPEG image I want. I did see a page on a chinese web site (lost the link but I'll find it & stick it on here or www.ianrolfe.com (http://www.ianrolfe.com) when) where someone has got it to work but that appears to be the only place :)
My next move it to dissect the linux driver (which doesn't seem to support JPEG) & see if I can combine it with rhe google translate of the website to get something working!
Ian
These people appear to have examples for JPEG from the 2640
http://www.arducam.com/ (http://www.arducam.com/)
Thanks Mike, I did have a look there but didn't find anything useful in my quick scan. I guess I need to dig a little deeper.
Ian
-
1 trillion frames??!?!? :wtf:
http://www.iflscience.com/technology/laser-pulse-imaged-through-bottle-1-trillion-frames-second (http://www.iflscience.com/technology/laser-pulse-imaged-through-bottle-1-trillion-frames-second)
1 trillion frames, very cool. They use equivalent sampling method to do this.
Photographic technique is not that hard overall (you need some very fast sensors, accurate timing triggers and laser source) - you could use very similar tactics to do that with any type of photo/video sensor arrays (for example, increase frame rate from 30FPS to 60FPS using 2 cameras in sync with 30FPS each).
What is the most interesting is not the technique, but the potential applications of such high speed imaging. Ultra sound echoscopy with light, see behind the walls, etc.
-
I noticed a recent piece in EE Times that seemed relevant to this thread:
GoPro Puts Consumers in 'Matrix'
http://www.eetimes.com/document.asp?doc_id=1323745& (http://www.eetimes.com/document.asp?doc_id=1323745&)
The gist of it is that Brit Timothy Macmillan of Time Slice Films was appointed by GoPro as "multi-camera development manager working on video arrays".
I presume GoPro is putting some development coin into "bullet-time"... and doing some PR to try to gauge what sort of commercial interest there would be.
-
I have a feeling that Mike is working on the 1st "Infra-red Bullet-time" effect camera rig using Flir Lepton sensors 8)