EEVblog Electronics Community Forum
General => General Technical Chat => Topic started by: EEVblog on July 05, 2013, 07:46:40 am
-
It's probably time I stopped using the cheap arse hosts for the podcast.
Looking at LibSyn who we use for the AmpHour show who are excellent and fast.
But their biggest plan at $75/month seems it would barely be enough at 1500MB.
For example, in the last 30 days I produced 15 episodes (9 hours 21 minutes worth of material) with a total length of almost 2GB in file size at 640x360.
I'll have to talk with LibSyn and see what they can do, but are there any other options I should consider?
And before everyone simply screams "Amazon S3" etc, do the math on the amount of outbound bandwidth required. That's 2TB of data every month for every 1000 viewers. I have at least 3500 regular podcast viewers. So we are talking towards the 10TB/month figure.
-
How about using torrents? That way you are using a peer-to-peer network to distribute the files at zero costs for you. Just offering this as an option may save you a serious amount of bandwidth (and costs).
-
What about a CDN like Limelight? http://www.limelight.com/services/orchestrate-video/video-platform-free-trial/ (http://www.limelight.com/services/orchestrate-video/video-platform-free-trial/) I guess it would cost similar to Amazon... not sure about that though.
-
How about using torrents? That way you are using a peer-to-peer network to distribute the files at zero costs for you. Just offering this as an option may save you a serious amount of bandwidth (and costs).
I already do, but ultimate the RSS feed must link to real file somewhere, and the majority will use that. Less than 50 people have chosen the torrent option.
-
https://www.frontrangehosting.com/hb/index.php/cart/cloud-servers/ (https://www.frontrangehosting.com/hb/index.php/cart/cloud-servers/)
10TB for $190USD per month, I use a $2/month hosting plan and $1/month VPS but not to the extent you'd would.
-
Just get a dedicated server with 100mbps unmetered bandwidth:
http://www.leaseweb.com/en/dedicated-servers/100mbps-unmetered-servers (http://www.leaseweb.com/en/dedicated-servers/100mbps-unmetered-servers)
http://www.ovh.co.uk/dedicated_servers/sp_16g.xml (http://www.ovh.co.uk/dedicated_servers/sp_16g.xml)
about 65$ a month for 100 mbps unmetered = ~ 30 TB a month
and you just ask your friend to set up a basic web server on it to serve files from.
-
Just get a dedicated server with 100mbps unmetered bandwidth:
about 65$ a month for 100 mbps unmetered = ~ 30 TB a month
and you just ask your friend to set up a basic web server on it to serve files from.
Sure, can do that, but one problem is that it's not a CDN, and is likely not optimised for massive user video streaming. What happens when 1000 people (or more) try to download the latest episode at once?
That's one advantage of a purpose designed CDN solution like Libsyn
-
Don't ya just love technology , it will screw us all one day
-
Just get a dedicated server with 100mbps unmetered bandwidth:
about 65$ a month for 100 mbps unmetered = ~ 30 TB a month
and you just ask your friend to set up a basic web server on it to serve files from.
Sure, can do that, but one problem is that it's not a CDN, and is likely not optimised for massive user video streaming. What happens when 1000 people (or more) try to download the latest episode at once?
That's one advantage of a purpose designed CDN solution like Libsyn
I will agree with you, a single dedicated server on a 100 mbps port will not handle beautifully 1000 people downloading at the same time. If 1000 will download at the same time, you'll get about 80-90 mbps split between 1000 users, obviously... each will download at 20-50 KB/s
You just have to ask yourself :
1. do you really think you're going to have 1000 users click on the download button at the same time?
The RSS feeds don't push the new video to all people at same time, most have the rss set to refresh once every 10-20 minutes or so, they don't all click on the download link the moment the update pops in the feed etc ... you'll probably have 1000 people spread out within about 20-30 minutes to an hour, making it much more manageable.
You could also do some tricks like using geolocation to delay the rss feed for some regions .. ex australia gets the rss post instantly, us gets it 10 minutes later, eu gets it 20 minutes later... this way you'll spread those 1000 users a bit... who cares when in the day user is notified of the video?
2. do you think the large majority of users will care whether the download speed is 5 MB/s or 100-200 KB/s?
A video is maybe 50-100 MB on average.. even at 100KB/s that video will download in 5-10 minutes which isn't that much. A lot of people have slow connections anyway so they won't really be bothered by the slow speed.
Youtube encodes videos at 360p and 480p at about 600-800 kbps.. that's about 80-110KB/s. Basically you could set rules in the web server to limit each user to about 150 KB/s maximum speed and you'd guarantee nobody has issues even if he/she streams the video directly from your server without downloading it in advance.
The apache server has filters already that you could configure for example to allow a burst of a few MB when user starts downloading the file and then if the server serves lots of users it can throttle down the connection to 100-150KB or more to keep everyone's download at a reasonable speed - this assures that if the person tries to stream the video directly he/she has a big chunk initially to watch.
Anyway... even if you want to think in advance, there are offers for dedicated servers with 1 gbps ports that aren't so expensive
In fact I plan to upgrade one of my servers at Swiftway to one of their latest offers published here:
http://www.webhostingtalk.com/showthread.php?t=1280239&highlight=swift (http://www.webhostingtalk.com/showthread.php?t=1280239&highlight=swift)
INTEL Xeon E3-1240 4 Core / 8 Threads
RAM: 16GB DDR3 internal memory
HDD: 2 x 128GB SSD or 2x WD 1TB 3.5 7200rpm 64MB SATA
RAID: Optional - see optional upgrades for details
Bandwidth: 1 Gbps port with 10TB monthly bandwidth package
IPMI 2.0 with Integrated KVM/IP with remote media support
Datacenter location: Netherlands or Chicago, USA
105$ for server with 1 gbps port with 10 TB included, 30$ for each additional chunk of 10 TB you add to the server.
Leaseweb offers something a bit more expensive and with lower performance hardware but with 100 TB of bandwidth at about 115$ and up : http://www.leaseweb.com/en/dedicated-servers/100-tb-servers (http://www.leaseweb.com/en/dedicated-servers/100-tb-servers)
Basically the catch is that it's a "volume network", meaning you may get up to 300mbps-800mbps depending on how loaded the network segment is, and each connection will average at about 400 KB/s to 2-3 MB/s depending where the user is ... their "premium" network can do even 10-30MB/s download speeds.
CDN is much more expensive than such offers because they're not really designed for what you want to do... a CDN's idea is to have the files you serve cached/spread out on various servers around the world and when user clicks on download, he gets the file from a file closer to him/her.
CDN is great to store javascript files, website images, logos, ads etc to keep a website responsive and fast... for videos.... use a cdn to serve a video ad so user won't feel like skipping it because it loads slowly... but the eevblog videos, I don't think speed matters that much.
I doubt these video downloads are that time sensitive and that viewers care so much about speed that you'd have to resort to such premium solution like a CDN.
-
Have you thought of download mirror sites?
I'd be happy to add your downloads to the list of projects I provide mirrors for (official php.net/CPAN/etc.) and I'm sure others probably would too.
Chris.
-
I will agree with you, a single dedicated server on a 100 mbps port will not handle beautifully 1000 people downloading at the same time. If 1000 will download at the same time, you'll get about 80-90 mbps split between 1000 users, obviously... each will download at 20-50 KB/s
That's still pretty slow.
1. do you really think you're going to have 1000 users click on the download button at the same time?
Yes. And there is also the continuous load of previous episodes.
2. do you think the large majority of users will care whether the download speed is 5 MB/s or 100-200 KB/s?
Quicker is always better.
I need to think of growth.
My limited numbers on the podcast at the moment are very likely because the performance sucks.
A video is maybe 50-100 MB on average.. even at 100KB/s that video will download in 5-10 minutes which isn't that much. A lot of people have slow connections anyway so they won't really be bothered by the slow speed.
The opposition leader in our country said something similar akin this in trying to support copper vs fibre. It's Bill Gates 640KB is enough for everyone all over again.
Youtube encodes videos at 360p and 480p at about 600-800 kbps.. that's about 80-110KB/s. Basically you could set rules in the web server to limit each user to about 150 KB/s maximum speed and you'd guarantee nobody has issues even if he/she streams the video directly from your server without downloading it in advance.
I don't want to rate limit people.
CDN is much more expensive than such offers because they're not really designed for what you want to do... a CDN's idea is to have the files you serve cached/spread out on various servers around the world and when user clicks on download, he gets the file from a file closer to him/her.
Umm, bingo!
That's what you want, the fastest possible for everyone. There is no way one dedicated box in one place on the planet will beat a distributed CDN.
That is why companies like LibSyn exist that use CDN's specifically for streaming podcasts.
but the eevblog videos, I don't think speed matters that much.
Speed is everything, it made a huge difference to our AmpHour show switching to LibSyn, and that's a lousy 30MB audio file.
-
Have you thought of download mirror sites?
I don't think mirrors work for RSS feeds? They need a fixed file location pointer.
-
Dave, it's always nice to have fast speeds but I was thinking of "good enough" for small amount of money.
As you can see my suggestions hovered around 100-120$ for up to 100 TB of transferred data at varying speeds and quality levels. CDN would get you more reliable speeds but at less overall bandwidth and higher price - if you can afford it go for it.
As for your question regarding rss feed and download mirrors .. you can easily set up a script which chooses automatically a download mirror for the user ...
For example you can have
eevblog.com/podcasts/download.php?file=100 or
eevblog.com/podcasts/download.php?file=EEVblog491-Nintendo64teardown-640x360.m4v
... and the script will automatically redirect browsers to one of the mirrors you previously set in the script. If you want I can write you such script in a couple of minutes.
-
Have you thought of download mirror sites?
I don't think mirrors work for RSS feeds? They need a fixed file location pointer.
At the fixed location a fairly simple redirect script using the MaxMind GeoLite database, a list of mirrors with the areas served etc. should help offload a good chunk of traffic.
-
How about BitGravity? It's what Revision3 uses for their video files. I couldn't find a price though, so you'll probably have to request a quote from them.
http://www.bitgravity.com/products/bg_video_delivery.html (http://www.bitgravity.com/products/bg_video_delivery.html)
EDIT: After a little more looking... Here's a list of a few CDNs: http://www.cdnplanet.com/cdns/ (http://www.cdnplanet.com/cdns/)
-
Wait, why is this linked from "one of my webpages was haxored"? are you blaming "cheap arse hosts" for garbahe unsecure php code you ran on calcwatch site?
btw was it php? how was the site hacked? Last time I looked at calcwatch it was some static page with few pictures - something that should be 100% secure.
-
A good host would not disable ALL websites hosted on the account, but only the affected domain and give you time to fix it before going all paranoid.
For example, years ago a friend of mine with a Dreamhost shared hosting account upgraded an image gallery script that turned out to be a cpu hog. The admins simply went in an added a php.ini in that folder which blocked the script from being processed, then notified the user.
-
If the files for all websites are accessible from the same account, then anyone with access to one of those sites could also make changes to the other sites. I think erring on the safe side is fair for a host, especially a cheap one. The burden is on the user to maintain a secure website, not on the host to be tolerant towards hacked accounts.
-
Funny, based on the "I need a new podcast host" I thought you were getting rid of Chris for a second. I haven't been keeping up to date with amp hours but I was about to check out the last one to see if you got into a fight or something :)
-
What do other videobloggers with similar number of views do?
Maybe you can start a cooperative with one of them to share the costs for a performant service?
-
Hi Dave
I have 2 HP DL380 servers idling in two of the data cabinets I maintain. Bandwidth not that much since we are in South Africa but I can offer them up for free to keep any part of the EEVblog going. Pop me a mail and we can talk specs if you have time.
-
What do other videobloggers with similar number of views do?
They do very same thing Dave does and upload their videos to YT/Twitch/whatever. I dont know what being a videoblogger has to do with hosting server when you DO NOT HOST ANY VIDEO FILES :)
I agree with alm - its users fault he used garbage hackable code on his account. Hosting stuff is not the same as posting updates on bookface.
-
What do other videobloggers with similar number of views do?
They do very same thing Dave does and upload their videos to YT/Twitch/whatever. I dont know what being a videoblogger has to do with hosting server when you DO NOT HOST ANY VIDEO FILES :)
Except Dave does host video files, hence the discussion about bandwidth and transfer caps...
-
What do other videobloggers with similar number of views do?
They do very same thing Dave does and upload their videos to YT/Twitch/whatever. I dont know what being a videoblogger has to do with hosting server when you DO NOT HOST ANY VIDEO FILES :)
Except Dave does host video files, hence the discussion about bandwidth and transfer caps...
Yes, I thought that this was the specific issue.
-
What do other videobloggers with similar number of views do?
They do very same thing Dave does and upload their videos to YT/Twitch/whatever. I dont know what being a videoblogger has to do with hosting server when you DO NOT HOST ANY VIDEO FILES :)
Except Dave does host video files, hence the discussion about bandwidth and transfer caps...
Does he? Where? :o
-
Force people to use torrents.
Give more alternatives to download with torrents .
Have a really cheap seedbox for your torrents.
Advertise your torrent downloads with good step by step wiki (download clients or browser addon) for standalone µtorrent, firefox (http://addons.mozilla.org/en-us/firefox/addon/bittorrent-surf-beta/)/chrome (http://chrome.google.com/webstore/detail/bittorrent-surf-beta/ibpbofogepkkeoockhkfcgngjkimndlp?hl=en-US) browser addon http://torque.bittorrent.com/oneclick/ (http://torque.bittorrent.com/oneclick/)
Use sites like http://www.bitlet.org/ (http://www.bitlet.org/) OR http://www.furk.net/ (http://www.furk.net/) OR http://zbigz.com/ (http://zbigz.com/) for people who don't want browser addons or software to download.
Donating bandwidth and storage space for your blog instead of the other donation methods would be nice. You could have your own CDN wherever people are interested in your videos.
I think people stay away from torrents because usually these kinds of files don't have many seeds and download rate really sucks compared to ftp/http from a good server.
Show people how good speeds can be with torrent downloads.
Torrents and other P2P are already responsible for majority of internet traffic today, why not join them at full force?
Make people download their videos from youtube (is it ok by youtube for you to do this?) there are already many youtube download sites and addons and programs and so on.
-
What do other videobloggers with similar number of views do?
They do very same thing Dave does and upload their videos to YT/Twitch/whatever. I dont know what being a videoblogger has to do with hosting server when you DO NOT HOST ANY VIDEO FILES :)
Except Dave does host video files, hence the discussion about bandwidth and transfer caps...
Does he? Where? :o
The downloadable podcast files - there's a .m4v link underneath each video on the blog itself, which are also the files that the RSS feed points at. Hence why Dave's host blocking BitLove stopped the BitTorrent RSS from working, etc.
-
I maintain a very simple website for a friend of mine. We use Hostmonster.
Hostmonster is OK for the sort of thing we do, but for more ambitious things, I'd look for a less mass-market site.
I'm also leery of peer to peer, but that may be just cause I'm an old fart and my trusting days are over.
-
Jupiterbroadcasting who produce a lot of video content on youtube use http://www.scaleengine.com/ (http://www.scaleengine.com/)
might be worth a look.
-
They do very same thing Dave does and upload their videos to YT/Twitch/whatever. I dont know what being a videoblogger has to do with hosting server when you DO NOT HOST ANY VIDEO FILES :)
Because for some reason a few thousand people seem to want to watch a compressed 640x360 of the video on their iPhone or whatever via their RSS reader.
Quite frankly I wish I didn't have to offer a podcast version it would save me a lot of trouble and now a fair bit additional cost.
-
Force people to use torrents.
I may as well save the trouble and force them to download/watch from Youtube.
-
OK, here's a stupid question: Why not just add some unrelated video and host them on youtube?
EDIT: Just saw your last comment - AFTER I posted. I still ask it, though. Why not? It's a worldwide CDN, it can handle all your traffic and more besides, you can make money from ads, and it's free.
-
Does he? Where? :o
There are 4 ways people can watch my videos:
- From within Youtube. The most popular.
- From the embedded Youtube player on eevblog.com 2nd most popular option
- From iTunes or some other RSS reader/player/viewer thingo. (3rd most popular). This option I have to host the video file myself, and to stop complaints I have to host all previous episodes as well. Maybe 3500 people take this option, but it's hard to get good stats because it can do multiple counts when your download fails because the connection is shit to my overtaxed server.
Not many video blogger do this because it's a PITA, and it can costs you a fortune to host the files when you become popular. I'm doubly screwed because my videos are very long.
- From the torrent feed, which currently still sucks off the same RSS feed.
-
And door number 3 is this:
1) Stop hosting any video files myself and adding any video link to my blog posts. Saving time, cost, and hassle. The RSS feed now contains just my text blog post and the embedded link code (which probably doesn't work for people)
2) Force everyone to use Youtube's RSS feed. My channel is here: my http://gdata.youtube.com/feeds/api/users/EEVblog/uploads (http://gdata.youtube.com/feeds/api/users/EEVblog/uploads)
I have not tried this myself, so will have to do that, but I don't use iTunes so have no idea if it works with that.
The downside of doing this is that:
a) everyone has to change their reader to point to youtube if they want to still get the videos that way.
b) continue to subscribe to the eevblog RSS Feed if they want my text blog posts and updates etc.
c) I don't know what happens to iTunes
I'm liking door number 3, but I will need people to test how well the Youtube RSS video feed works
-
I haven't seen it used hardly at all lately, but Coral still seems to be running. Worth a try, perhaps backed by a cheap unmetered dedicated server?
http://www.coralcdn.org/ (http://www.coralcdn.org/)
Edit: Ah nevermind it looks like they've had to cap filesize at 50MB. As far as door number 3, you could easily proxy or redirect the existing feed URL to the Youtube feed. Users would probably get a pile of 'new' posts they'd already read, but other than that shouldn't have to change anything.
-
...
2) Force everyone to use Youtube's RSS feed. My channel is here: my http://gdata.youtube.com/feeds/api/users/EEVblog/uploads (http://gdata.youtube.com/feeds/api/users/EEVblog/uploads)
I have not tried this myself, so will have to do that, but I don't use iTunes so have no idea if it works
...
I'm liking door number 3, but I will need people to test how well the Youtube RSS video feed works
Tried it with the link you supplied on an iPad with latest version of IOS and podcast viewer:
Fail, error:
Unable to subscribe
The link provided is not a valid podcast.
-
and when you add that url to iTunes (11.0.2.26) on a windows 8 pc, it shows a podcast with 0 casts.
-
and when you add that url to iTunes (11.0.2.26) on a windows 8 pc, it shows a podcast with 0 casts.
Hmm, thanks.
Work great with Miro for me.
Can you try this one:
https://gdata.youtube.com/feeds/base/users/eevblog/uploads
-
Here are some methods for iTunes:
http://lifehacker.com/how-to-turn-youtube-channels-into-subscribable-podcasts-496976976 (http://lifehacker.com/how-to-turn-youtube-channels-into-subscribable-podcasts-496976976)
Although I will have to change the official feed on my iTunes page to try and read the Youtube feed.
-
Can you try this one:
https://gdata.youtube.com/feeds/base/users/eevblog/uploads
Fails with different error on iPod:
Unable to subscribe
The supplied link is not a valid podcast
Also tried with http in stead of https. No difference.
-
When I read the title, I thought you were getting rid of Chris lol
-
My limited numbers on the podcast at the moment are very likely because the performance sucks.
..or maybe people prefer Youtube...
There are 4 ways people can watch my videos:
However many different ways you choose to make content available, someone will use it. I'm sure if you offered to post out DVDs once a month a few people would use it.
The question is how many users of a particular distribution method would give up watching and not use another method , and is it worth the effort to support those people?
I know there will be a few people with quasi-religious objections to certain channels, but at some point you need to make a cost/effort vs. benefit choice.
I can see that it is important to offer some way to download for offline viewing, but it seems that there are ways to do this off Youtube, and presumably itunes isn't costing you anything for hosting. How many people would you lose if these were the only outlets?
Think about the possible tradeoff between user loss due to more limited distribution vs. gain due to you having more time to make more content.
I have no idea what the answer is, and our field is probably too specialist for mainstream stats to be meaningful (also most potential subscribers will be more tech-savvy than potential viewers of fluffykittens.com) , but one thing I do know is I've never had a single request from anyone to make my stuff available outside YouTube.
-
I prefer you have 30 minute more for produce content and 30$ more for your famili or the opposite, instead of spend it on not-so-useful distribution metod
-
The torrent option is basically free, and could even be managed by volunteers if all else fails. The question is not if the torrent should be kept but whether that should be the only option to get an offline download. The real question is if the http/CDN option is needed for mobile devices any more. The difference is just convenience, IMO. Torrenting is cheap but a "pro user" option. I think it should especially kept for getting a high res copy of certain videos (teardowns and such) which look especially pornographic in 720p or 1080p. (But for myself, I can just dump them from YT and play them in VLC, which I find plays HD content better than the web browser on my crappy computer.)
In theory, everyone has a mobile device which can play YT videos and has cellular internet connectivity. In practice, not everyone does, and the internet connectivity might be crap or you might be limited by your data plan. But as time progresses I think the need for an option between YT and torrents will approach zero. Another option might be to educate users on how to download videos from YT for offline use. YT already offers encodings in various qualities, it's just a matter of using the right tool to download them.
-
..or maybe people prefer Youtube...
Yes.
But it is also undeniable that a huge percentage of people now use their mobile device to watch video.
So it's a matter of which mechanism they use to do that.
However many different ways you choose to make content available, someone will use it. I'm sure if you offered to post out DVDs once a month a few people would use it.
The question is how many users of a particular distribution method would give up watching and not use another method , and is it worth the effort to support those people?
I know there will be a few people with quasi-religious objections to certain channels, but at some point you need to make a cost/effort vs. benefit choice.
Well, yes, I've always been of that opinion when it comes to the podcast version. It's always been that PITA thing I have to maintain.
I've kept it going because at least a few thousand do seem to use. That's a large number, but realistically is only 10% or less of my audience.
And there are ways many of them can keep doing that without me having to host the files.
I can see that it is important to offer some way to download for offline viewing, but it seems that there are ways to do this off Youtube, and presumably itunes isn't costing you anything for hosting. How many people would you lose if these were the only outlets?
iTunes do not host anything. I do not and cannot upload anything to iTunes (at least last time I checked). It works by sucking your RSS feed and the pointer you give it to where the video file is located.
Think about the possible tradeoff between user loss due to more limited distribution vs. gain due to you having more time to make more content.
Yes, that's what I'm doing, I am very seriously considering dropping hosting the file myself.
but one thing I do know is I've never had a single request from anyone to make my stuff available outside YouTube.
I get requests all the time for how to download the files directly. They have no idea what RSS is or that I have a website with all the files, albeit at 640x480 or less.
-
The torrent option is basically free, and could even be managed by volunteers if all else fails. The question is not if the torrent should be kept but whether that should be the only option to get an offline download.
It's not. There are dozen diffeent ways to rip the video from Youtube, or properly download it via the RSS feed I gave above.
In theory, everyone has a mobile device which can play YT videos and has cellular internet connectivity. In practice, not everyone does, and the internet connectivity might be crap or you might be limited by your data plan.
This is precisely why people use iTunes and/or RSS readers/players. They grab your content when online at home and then auto dump the file to your mobile device so you can playback seamlessly anywhere. Only a fool would chew up a few hundred MB of expensive mobile internet bandwidth to watch a single hour long youtube video while on the move.
Another option might be to educate users on how to download videos from YT for offline use.
Yep, I think I have to finally say, sorry, that's it. My videos are only available on Youtube. Here are some links to a dozen ways to download/feed/rip/iTunes it
-
the feed doesn't seem to have any kind of content that a generic podcast reader can access. Even in a browser there's no attachment.
Where's the link to those ways to get the files from YT? :-//
-
the feed doesn't seem to have any kind of content that a generic podcast reader can access. Even in a browser there's no attachment.
It works 100% for me using Miro.
It downloads a 1280x720 version of the video no problems.
:-//
-
I watch you 100% from ipad, 50% offline, 100% from youtube
Just i use one of the 8227282829292929 apps for that
-
Does he? Where? :o
The downloadable podcast files - there's a .m4v link underneath each video on the blog itself, which are also the files that the RSS feed points at. Hence why Dave's host blocking BitLove stopped the BitTorrent RSS from working, etc.
Holy crap you are right. Well this is weird/awkward.
I know of only one Video podcast that does this (Hak5) and they use their parent company (revision3, internet video producer behemoth) for hosting. Hosting on your own is just crazy and expensive.
-
is there really anybody interested on how dave organizes his shit?
i mean really.. let the grown up deal with it and let him announce
us where we can follow his fantasies.
hopefully online.. so we can watch over and over how you fucking
dislike cats beeing youtube selfsutained .. you know.. selling
t-shirts, industry bitch and ways to be succesful in dating sites..
hahahah
what a looser.. :-DD
-
The torrent option is basically free, and could even be managed by volunteers if all else fails. The question is not if the torrent should be kept but whether that should be the only option to get an offline download.
It's not. There are dozen diffeent ways to rip the video from Youtube, or properly download it via the RSS feed I gave above.
In theory, everyone has a mobile device which can play YT videos and has cellular internet connectivity. In practice, not everyone does, and the internet connectivity might be crap or you might be limited by your data plan.
This is precisely why people use iTunes and/or RSS readers/players. They grab your content when online at home and then auto dump the file to your mobile device so you can playback seamlessly anywhere. Only a fool would chew up a few hundred MB of expensive mobile internet bandwidth to watch a single hour long youtube video while on the move.
Another option might be to educate users on how to download videos from YT for offline use.
Yep, I think I have to finally say, sorry, that's it. My videos are only available on Youtube. Here are some links to a dozen ways to download/feed/rip/iTunes it
Agree, don't even try hosting video yourself at scale, unless you have buckets of money to throw at the problem to get returns of scale, its a mugs game.
Given the 'grunt' available on mobiles with the Gb's of local storage plus tonnes of rip to go app's, the vast majority of those on RSS should be able to move across without headache. Just provide some pointers.
BTW in Cambodia atm - you should have a few more followers..
-
Is it ok for youtube if you tell people how to download your videos from them, so you don't get any extra cost for distributing videos? Youtube works by advertising, if people just download your videos from youtube, they don't get to show their adverts. I thought "YouTube makes you agree to view videos through the means provided on the YouTube website." anything other than that is a grey area. Are you biting the hand that feeds you?
Whats so wrong about torrents? Content distribution with P2P works really well when there are enough peers. There are no ready made services which handles this automatically for you? I think we can work something out.
-
If we download YouTube videos with the RSS feed, will those video's contain text overlays that Dave adds? I thought that those were added (i.e. burned into) to the downloaded video m4v files and were captions when viewed in YouTube's web player?
One other thing... and I think it was mentioned above. The YouTube RSS feed doesn't have any enclosure tags with the file links. This makes it so that regular RSS readers can't download the video files. Is it possible to add the enclosure tags with links to the video files to the YouTube feed? I'm not sure how I will be able to download the files without the enclosure tags.
-
Whats so wrong about torrents?
Nothing.
But I've tried it and maybe 20-30 people used it. It's not worth bothering with.
-
If we download YouTube videos with the RSS feed, will those video's contain text overlays that Dave adds? I thought that those were added (i.e. burned into) to the downloaded video m4v files and were captions when viewed in YouTube's web player?
One other thing... and I think it was mentioned above. The YouTube RSS feed doesn't have any enclosure tags with the file links. This makes it so that regular RSS readers can't download the video files. Is it possible to add the enclosure tags with links to the video files to the YouTube feed? I'm not sure how I will be able to download the files without the enclosure tags.
I don't know the in's and out's out it, but Miro worked fine and downloaded the file without any text overlays or ads or hassle, it just worked.
-
Dave,
About the overlay comment I made... I meant when you make a correction to what you said or when you add more detail on a part. For example, in some videos you'll say something like: "I'm not sure what that part is." After you make the video you've looked it up and added a text overlay that says something like "(It's a high speed DAC)".
I thought that in the YouTube videos, you used the captioning/annotation feature so the "(It's a high speed DAC)" message isn't in the actual video, but then you did add it to the downloadable videos in the actual video itself.
If I download the YouTube videos, will I get the overlay text that you sometimes add? Sorry if I wasn't clear the first time. :)
-
Dave,
I understand that the extra distribution methods can be a hassle for you. If you think that moving over to exclusive Youtube distribution is the only way forward, then I respect that it is your choice to do so. However, as a user of the HTTP downloads, I thought I would give you my perspective on the issue.
If I were in your position Dave, I would think carefully before relying on Youtube for distribution. The idea of having one central server (even if virtual/cloud-based/whatever) for everything obviously gives a single point of failure, and goes against the 'peer-to-peer' design of the Internet. Back in the days of static personal homepages, with lower bandwidth requirements, this idea was alive and well, and it is only with the rise of video streaming that we have seen a move away from this.
Although a technical failure of a site as large as Youtube may seem remote, another, potentially much bigger problem is that you are at the mercy of whatever business policies they decide to implement. It is not too hard to imagine someone taking offence at a video, and applying pressure to Youtube, resulting in a video, or even an entire series, being pulled down. Or Youtube could decide there are too many liability issues in encouraging people to pull things apart, or they could add even more obnoxious advertising, etc.
While I admit that some of these concerns also apply to a privately hosted website, at least then you have the option of shopping around for different web hosts.
Personally, I tend to stay away from Youtube as much as possible. The site is large and bloated, and they deliberately make it difficult to download the videos for viewing later.
Even if the bandwidth requirements for hosting the files via HTTP are excessive, I would be in favour of keeping the torrents, as these are really an ideal means for video distribution. I did try using them before, however they did lag behind considerably in terms of updates. If this could possibly be addressed, I am sure quite a few people would take up this distribution option.
Anyway, whatever way you go, I am sure I will keep watching the show, even if it means I have to switch from wget to youtube-dl for downloading it. :)
-
Well, until now i didn't even know that the videos could be downloaded via something else than Youtube. Since my cable sucks sometimes, i had to start using YTD Youtube downloader. This has not failed me with once for getting the EEVBlog videos. This downloading has also a positive other effect : I can now watch the video on my Media Center with the big TV connected to it.
-
I'm using this Firefox extension :
Download Youtube as MP4 : https://addons.mozilla.org/en-US/firefox/addon/download-youtube/ (https://addons.mozilla.org/en-US/firefox/addon/download-youtube/)
It's also available as a Greasemonkey script : http://userscripts.org/scripts/show/25105 (http://userscripts.org/scripts/show/25105)
It basically adds a download button under the video with a drop down menu from which you can select the format and quality and then it basically downloads the video using Firefox's built in download manager as a regular file.
No conversions, no quality loss, allows you to get the format you like.
Sometimes if the video starts as 360p or 480p you have to select 720p or 1080p in youtube's player and reload page to get the HD versions in the combo box but that's just a page reload..no biggie
-
Dave, I don't think you should advertise and tell people to use extensions to download the videos, it might be against the TOS...
Under some condition you can add a download button under videos, but i think you have to put them under an open source type license after uploading and that may be an issue with the advertising.
-
If I were in your position Dave, I would think carefully before relying on Youtube for distribution.
Almost every single full time Youtuber on the planet does just that.
another, potentially much bigger problem is that you are at the mercy of whatever business policies they decide to implement. It is not too hard to imagine someone taking offence at a video, and applying pressure to Youtube, resulting in a video, or even an entire series, being pulled down. Or Youtube could decide there are too many liability issues in encouraging people to pull things apart, or they could add even more obnoxious advertising, etc.
Those risks exist regardless of what I do with the podcast.
Don't worry, I can instantly re-enable uploads if I need to.
Personally, I tend to stay away from Youtube as much as possible. The site is large and bloated, and they deliberately make it difficult to download the videos for viewing later.
Youtube is practically 100% responsible for the successful growth of my blog.
-
Is it ok for youtube if you tell people how to download your videos from them, so you don't get any extra cost for distributing videos?
They support RRS feeds:
https://www.youtube.com/t/rss_feeds (https://www.youtube.com/t/rss_feeds)
-
If I download the YouTube videos, will I get the overlay text that you sometimes add? Sorry if I wasn't clear the first time. :)
If it's an annotation on Youtube, no idea.
If it's text I add in the video editing process, then yes.
-
Those risks exist regardless of what I do with the podcast.
Don't worry, I can instantly re-enable uploads if I need to.
OK, fair enough. I will have a look at the RSS feeds.
-
I know it is off-topic for this thread but I wanted for a long time to rant about hostmonter (under all incarnations as they use a few different names). They would indeed disable the account out of the blue and generaly just working hard not to give you the service.
-
Do you have hard numbers on what the highest concurrent load is now?
-
I'm using this Firefox extension :
Download Youtube as MP4 : https://addons.mozilla.org/en-US/firefox/addon/download-youtube/ (https://addons.mozilla.org/en-US/firefox/addon/download-youtube/)
It's also available as a Greasemonkey script : http://userscripts.org/scripts/show/25105 (http://userscripts.org/scripts/show/25105)
Sometimes if the video starts as 360p or 480p you have to select 720p or 1080p in youtube's player and reload page to get the HD versions in the combo box but that's just a page reload..no biggie
Switch to yousabletubefix
http://userscripts.org/scripts/show/13333 (http://userscripts.org/scripts/show/13333)
you will be able to configure default resolution - no need to switch and reload pages
-
from reading the post on the home page i thought id better let u no that it killed the itunes subscription
-
I expect my view is a minority one, but personally I care quite a lot that the podcast is available automatically in the iTunes Music Store, but I don't care a jot if it arrives there a day or two after it's published elsewhere.
Although it's not a perfect experiment, I watch most of the EEVblog episodes, but very few of other similar ones e.g. Mike's Electric Stuff. I think the main difference is iTMS availability.
-
The same happens to me. Until now (ie, before my ipad was stolen) the only way I watched the eevblog was via itunes. This is just because I travel a lot and use that time to watch the videos.
Sooner o later I will have a mobile device again and as Dave mentioned before the process is to download the content while online and sync and view it while offline. It is the trend so I think that, whatever the final method of content distribution Dave finally adopts, this should be taken into consideration.
It should be an easy and multi platform solution.
Charlie
@CHexclaim
-
There are two options that I know of and have used for this in the past.
1) SoftLayer's CDN. Not sure on the pricing as generally my clients just hand me the login info to configure it for them, but there is an added benefit here that Dave's server with HostGator is actually provided by SoftLayer, so it will (in theory) have a quick uplink into the CDN.
2) Amazon S3. I really like this service as it is very cheap for mass offsite storage, great for backing up servers using automated means, and it is pay as you go, so you only pay as much as it is used.
$0.095 per GB for storage.
$0.000 per GB for the first 1GB of data transfer out
$0.120 per GB for the first 10TB of data transfer out (this will possibly be what this site will be under)
$0.090 per GB for the first 40TB of data transfer out
$0.004 per 10,000 GET requests
So let us assume 2000 downloads a month,video is what 200MB max (I have no idea as I have never downloaded them).
2000 * 200MB / 1024 = 390.625GB * 0.120 = $46.875 + $0.004 = $46.879
So that works out to roughly 2.4c per download and it is backed by Amazon's massive CDN.
-
So let us assume 2000 downloads a month,video is what 200MB max (I have no idea as I have never downloaded them).
2000 * 200MB / 1024 = 390.625GB * 0.120 = $46.875 + $0.004 = $46.879
So that works out to roughly 2.4c per download and it is backed by Amazon's massive CDN.
Try 2000 downloads a DAY, not a month!
-
If I download the YouTube videos, will I get the overlay text that you sometimes add? Sorry if I wasn't clear the first time. :)
If it's an annotation on Youtube, no idea.
If it's text I add in the video editing process, then yes.
YouTube annotations are stored in a separate XML file: https://www.youtube.com/annotations_invideo?features=1&legacy=1&video_id= (https://www.youtube.com/annotations_invideo?features=1&legacy=1&video_id=)<video_id>
-
Wonder if Cloudflare might be able to handle ya... you can use the host you want, and hopefully cloudflare will keep you from sucking up all your bandwidth by acting a lot like a CDN.
https://www.cloudflare.com/overview (https://www.cloudflare.com/overview)
https://www.cloudflare.com/features-cdn (https://www.cloudflare.com/features-cdn)
Shoot them an email. They're extremely responsive anytime I have questions, problems, etc. Bet they could handle ya. =) And they don't charge for bandwidth! Oh, and the best part, it's all auto-magic. You just let them handle your DNS, and everything routes through them, they cache stuff as it wizzes by, and Bob's your uncle (hey, I have an uncle Bob!).
-
Only a fool would chew up a few hundred MB of expensive mobile internet bandwidth to watch a single hour long youtube video while on the move.
:-[
Oh well, if I don't use it for Dave's videos what will I use it for?
-
Only a fool would chew up a few hundred MB of expensive mobile internet bandwidth to watch a single hour long youtube video while on the move.
I get unlimited transfer on my phone.
One could say only a fool would assume all providers rip off their customers equally.
-
Only a fool would chew up a few hundred MB of expensive mobile internet bandwidth to watch a single hour long youtube video while on the move.
I get unlimited transfer on my phone.
One could say only a fool would assume all providers rip off their customers equally.
everybody gets ripped off by their providers, just in other ways.
most "unlimited" plans let you transfer X amount of data at full speed, then throttle you down for the remainder of your billing period. a "FAIR USEAGE" clause if you will.
I would be surprised if your company didn't do this to you, even if your plan does not specify throttling will occur.
-
Only a fool would chew up a few hundred MB of expensive mobile internet bandwidth to watch a single hour long youtube video while on the move.
I get unlimited transfer on my phone.
One could say only a fool would assume all providers rip off their customers equally.
everybody gets ripped off by their providers, just in other ways.
most "unlimited" plans let you transfer X amount of data at full speed, then throttle you down for the remainder of your billing period. a "FAIR USEAGE" clause if you will.
I would be surprised if your company didn't do this to you, even if your plan does not specify throttling will occur.
I haven't reached that point and I've on occasion pulled more than 10GB in a day. We aren't all stuck with the services found in certain countries.
Now, roaming, that is a ripoff..
-
(Already posted condensed on the blog, but this is a better place to discuss)
OK, here my thoughts about the topic: I understand your hassle with local hosting but still think this is a step backwards. You would move away from a open, more or less decentral system with documented protocols to some properitary, only sparse supported platform. In fact swtiching to YouTube would - in the first place - prevent me from viewing your Videos. Usually i let my phone download the videos over night via WiFi so i can watch them locally the next morning on public transport. Longer videos i watch on my TV which streams them based on the RSS.
Yes, there is a RSS-Feed (ish) thing on YouTube, but this is just text with a link to the YouTube website - no embedded file, no valid video podcast and as such not useable in many programs (yes, some do, but YT is constantly changing their site which breakes this feature). Also keep in mind: Google already dropped RSS in most of their products so i would not rely on this interface. And yes, there are ways to download from youtube, but its a PITA to automate and only supported on a small number of systems (mostly just on PCs) - i guess neither my mobile phone nor my TV would support this.
And yes, a lot of (video) podcasts also dropped their hosting - i stopped following most of them because i simply can not use their new streaming solutions on my devices and it is too labor-intensive to keep up manually. I only hacked something together for a few of them, but most remaining in my list are the Revision3 and CCC ones which still serve files.
In the end its of course your decision and since your content is worth it i would take the hassle and set up a local mirror of your files grabbed from YT to regain access from my phone/TV but i still think its a step backwards for the "open internet".
-
But do you have any suggestions of how to solve the problem of a ridiculous amount of traffic without breaking the bank?
-
have seen that the itunes feed is up and runing but is not having the new videos put on it will the itunes ever be updated again :wtf:
-
Note to Dave or the creator of the XBMC EEVblog app.
Please switch the RSS feed source to the youtube feed. The EEVblog app on my RaspBMC is recently (~4 months) going to crap. The youtube app always has the most recent videos lately. Certain episodes in the current app show up with a description yet will not play. There is also a huge delay in the new videos showing up in the EEVblog app, sometime days it seems.
Dave you look great on the 60" LCD TV. Having to dick around with the youtube app is a shame as the EEVblog app is super simple to navigate thought the TV compared to the youtube app.
-
I don't know anything about the XBMC app, it's not mine and have never used it.
-
I follow Jason Scott on twitter (@textfiles) and the other day he was looking for people who wanted to host their stuff with the Internet Archive. He specifically mentioned podcasts. May want to see what he has to offer?
-
They would have to be released under a permissive license (open source, creative commons, permission to freely to distribute etc)... and that Jason Scott is a scumbag, I've dealt with him in the past and he's bringing down the relatively good name of Internet Archive.
I'm paying about 100$ a month for a dedicated server with 100 mbps unmetered bandwidth (30TB a month) for years now. Costs the same to have 10 TB a month on a 1 gbps port. Complaining about bandwidth is just stupid these days.
-
dedicated server with 100 mbps unmetered bandwidth (30TB a month)
Complaining about bandwidth is just stupid these days.
unmetered 30TB? and you call other people stupid? :)
-
A computer connected to a 100 mbps switch port can transfer about 30-33 TB within a month in both directions.
100 mbps = 100.000.000 bits / 8 bits = 12,500,000 bytes /s = 12.207 KB/s = 11.92 MB/s
11.92 MB/s
x 3600s in an hour = 42,912 MB
x 24 hours in a day = 1,029,888 MB
x 31 days in a month = 31,926,528 MB = 31,178.25 GB = 30.44 TB
Realistically, with TCP/IP overhead, data packets and so on, you can expect about 28-29 TB of real data transfers in a month.
So what I was saying is that the dedicated server I rent is unmetered but the switch it's connected to has a 100 mbps port. Hence the 30 TB figure. If I want to I can ask to have the server moved to a 1 gbps port, but I'd pay extra if I go over 10 TB a month.
Here's a bandwidth graph:
(http://savedonthe.net/image/1874/traff.png)
-
yeah im the stupid one :o
http://www.wolframalpha.com/input/?i=100mbps+ (http://www.wolframalpha.com/input/?i=100mbps+)*++month
still if they charge above 10TB I suspect they will throttle if you cross it on 100mbit
-
No, I've had months when I had a sustained 60-80 mbps for weeks.
A company buys bandwidth for around 0.8-1$ a megabit, in volume... let's say $900 for 1 gbps 95 percentile. But keep in mind this company has servers in a large European datacenter and a large percent of the traffic is passed through free peering links with various other European datacenters so that traffic is almost free.
It's easy to put 30-35 servers in a 42U rack with a switch linked to the core router through a 2gbps (pair of standard 1gbps cables) or a 10 gbps fiber.
The company can give 10 TB on 1 gbps for the same price as 100 mbps unmetered because the user would know to be careful not to push 1 gbps all the time (10 TB is basically an average of 30 mbps throughout the month), the excess is something like 1-2$ per 10 GB or something like that (unless you mail them and choose to pay something like 30$ for extra 20 TB).
The real 1gbps unmetered links (330 TB a month) are about 400-600$ a month for bandwidth alone at this company. It's a gamble on their part, they gamble a large percent won't saturate their gigabit links and eat into their profit. For such bandwidth usage, you'd need a server with good I/O so that means either lots of RAM, or lots of drives, or SSD drives.. and renting such server cost money and have a higher profit margin compared to cheaper servers.
They offer (or used to offer a few months ago) some cheaper 1 gbps unmetered links for around 2-300$ a month but you have to agree not to rent such servers and use them for some particular cases like CDN, radio streaming, youtube style sites etc because the uplink can be saturated easily and if you push 1gbps for long time it affects other servers in the rack. On average, you get 200-500 mbps at any time, so it's still good value for money if you need a server that has occasional bursts of traffic (for example Dave posting a new video every few days, or the amphour show, or pushing a patch for a free software you develop etc etc)
Hosting your own podcasts on such server or offering the youtube video collection to download etc would totally be OK.
ps. If anyone cares the company is Swiftway ... the offers on their sites aren't that great, see the WHT offers here if anyone's curious : http://www.webhostingtalk.com/showthread.php?t=1306222&highlight=swiftway (http://www.webhostingtalk.com/showthread.php?t=1306222&highlight=swiftway)