As an avid member of the selfhosting community, I actually have a lot of familiarity with this, since tools for downloading, playing and streaming media are a very popular thing to self host.
So my understanding - entirely third hand of course - is that the following setup is used very widely:
1a. A torrent client such as Transmission or Deluge is run inside of a Docker container alongside a VPN client. Because of the nature of containers, the torrent client is unable to access the
internet at all if the VPN isn’t running. Torrent clients are, of course, intended for allowing users to share media that they legally can access and share, but their underlying technology unfortunately offers no way to verify that.
1b. Either alternatively, or in addition, a usenet client is set up in the same way. I’m a little less familiar with how usenet works to be honest.
Next, the user sets up a “Servarr” stack. This consists of multiple different self hosted apps, all of which can be run very easily using Docker. These programs scan and monitor the user’s entire media collection, and then automatically download new episodes of watched shows, download movies when an appropriate release is available (the user might, for example, configure to ignore cam releases and wait for digital). I’m told they even have calendars to show when content will be available. The user adds everything as easily as searching for the name of a show. It’ll even automatically pick up new episodes and seasons as they air. Servarr sends all its download requests to the torrent or usenet clients, and then moves the files into an appropriate storage folder when they are ready. It also monitors existing files and replaces them when better quality releases are found.
2b. (For those unfamiliar, Docker is a kind of all in one platform for running lots of server apps together on the same PC, configured using very simple config files called Compose files. The Compose file contains all the settings for the app in one place, and Docker runs it in a way that prevents it from conflicting with, or interacting with, anything else on your system).
Finally Plex or Jellyfin are setup (probably through Docker again, for convenience) to allow all the media to be streamed wherever the user is, just like having your own personal Netflix. Of course, these programs were designed to allow users to stream their own legally owned media to themselves only, but there’s no real way to control what users load into them. From there the files can be viewed easily through a web interface or a multitude of apps for phones, tablets, smart TVs and so on. Servarr has hooks that automatically inform Jellyfin or Plex to update their media libraries when new files, are added.
Optionally, something called Ombi is added to this setup. It’s a little helper program that allows people to request media be added to the download / watch queue without needing access to any of the backend management (say, for giving family members an easy way to add media to the system). It even includes an optionally approval system so the owner can have final say over what is being added. I imagine this is very useful for people with children, or who are giving access to friends, for example.
Edit: I should note for completeness that the hardware requirements for all of this are quite minimal. Typically a Raspberry Pi 4 or similar is used, with a USB hard drive or a NAS for storage.
The net effect of all this is that a new episode of The Boys goes up on Amazon and almost the same afternoon it’s there on the Jellyfin front page, waiting to be watched. I’m told the convenience is, if anything, higher than that of using streaming services as all the media arrives in one place (I’d imagine thats a huge plus to kids and less technical family who don’t enjoy having to remember which service a particular show is on, especially with some services renaming themselves constantly).
Of course, someone could also purchase blu rays of movies and shows, convert them to media files using a program like Handbrake or MakeMKV, and then add those to their Jellyfin or Plex library. Of course getting good conversions is fiddly, so maybe better to just buy the blu rays and then download a file that someone else has ripped for you already. Much less hassle that way, and you still legally own a copy. But I shouldn’t be editorializing here, like you said this is entirely academic. And who knows, buying a blu ray of Wish probably also gives Disney a legal right to murder you according to their lawyers.
1b. Either alternatively, or in addition, a usenet client is set up in the same way. I’m a little less familiar with how usenet works to be honest.
I can add a little bit of context to the Usenet side of things. I’m by no means an expert, but I at least know the surface level stuff.
Usenet is more like a server dead-drop. Usenet providers host servers, and people upload content to those servers. Then anyone who also has access to those Usenet servers can swing by and grab a copy. You use a Usenet reader to actually browse and download the files, sort of like how you use a torrent client to download torrents. The upside is that you don’t need to worry about whether or not a torrent is properly seeded; There’s a dedicated server that is hosting the file. The big downside here is that you actually need access to those servers. This requires a Usenet subscription, the same way you subscribe to your ISP for internet service. Not all Usenet providers have access to every server, (though many providers make attempts at parity with partnered companies.) So it’s common to need more than one Usenet subscription, for access to multiple servers for your various types of media.
Usenet subscriptions typically work in one of two ways; By usage, or by time. Usage is like a prepaid cell phone. Maybe you buy 100GB, and you can then download 100GB from the usenet servers before needing to pay again. These tend to be cheaper in the short term, but more expensive if you’re downloading tons of data constantly. Then there are the timed subscriptions, which are just like a subscription you’d expect; You pay for a month, and you have access for a month. Many people will keep a monthly subscription with their main provider, then a usage subscription with a backup. So even if their main doesn’t have a file, their backup might, and they’re only paying for the backup when their main is failed.
If this sounds similar to cloud server sites like Mega or Google Drive, that’s because it is. And it suffers from the same hurdles; Content owners can issue DMCA takedown notices on the media they own, and force the Usenet provider to remove it from their server. Usenet has historically been a more solid way to find full file downloads, but that was largely because content owners hadn’t bothered checking them for potential takedowns. In the past few years that has changed, and files often get taken down shortly after they’re posted. If a download fails on usenet, it’s often because the file got taken down while you were downloading it.
As an avid member of the selfhosting community, I actually have a lot of familiarity with this, since tools for downloading, playing and streaming media are a very popular thing to self host.
So my understanding - entirely third hand of course - is that the following setup is used very widely:
1a. A torrent client such as Transmission or Deluge is run inside of a Docker container alongside a VPN client. Because of the nature of containers, the torrent client is unable to access the internet at all if the VPN isn’t running. Torrent clients are, of course, intended for allowing users to share media that they legally can access and share, but their underlying technology unfortunately offers no way to verify that.
1b. Either alternatively, or in addition, a usenet client is set up in the same way. I’m a little less familiar with how usenet works to be honest.
2b. (For those unfamiliar, Docker is a kind of all in one platform for running lots of server apps together on the same PC, configured using very simple config files called Compose files. The Compose file contains all the settings for the app in one place, and Docker runs it in a way that prevents it from conflicting with, or interacting with, anything else on your system).
Finally Plex or Jellyfin are setup (probably through Docker again, for convenience) to allow all the media to be streamed wherever the user is, just like having your own personal Netflix. Of course, these programs were designed to allow users to stream their own legally owned media to themselves only, but there’s no real way to control what users load into them. From there the files can be viewed easily through a web interface or a multitude of apps for phones, tablets, smart TVs and so on. Servarr has hooks that automatically inform Jellyfin or Plex to update their media libraries when new files, are added.
Optionally, something called Ombi is added to this setup. It’s a little helper program that allows people to request media be added to the download / watch queue without needing access to any of the backend management (say, for giving family members an easy way to add media to the system). It even includes an optionally approval system so the owner can have final say over what is being added. I imagine this is very useful for people with children, or who are giving access to friends, for example.
Edit: I should note for completeness that the hardware requirements for all of this are quite minimal. Typically a Raspberry Pi 4 or similar is used, with a USB hard drive or a NAS for storage.
The net effect of all this is that a new episode of The Boys goes up on Amazon and almost the same afternoon it’s there on the Jellyfin front page, waiting to be watched. I’m told the convenience is, if anything, higher than that of using streaming services as all the media arrives in one place (I’d imagine thats a huge plus to kids and less technical family who don’t enjoy having to remember which service a particular show is on, especially with some services renaming themselves constantly).
Of course, someone could also purchase blu rays of movies and shows, convert them to media files using a program like Handbrake or MakeMKV, and then add those to their Jellyfin or Plex library. Of course getting good conversions is fiddly, so maybe better to just buy the blu rays and then download a file that someone else has ripped for you already. Much less hassle that way, and you still legally own a copy. But I shouldn’t be editorializing here, like you said this is entirely academic. And who knows, buying a blu ray of Wish probably also gives Disney a legal right to murder you according to their lawyers.
That’s appalling. Thank you for the instructions. I mean “warning”.
I can add a little bit of context to the Usenet side of things. I’m by no means an expert, but I at least know the surface level stuff.
Usenet is more like a server dead-drop. Usenet providers host servers, and people upload content to those servers. Then anyone who also has access to those Usenet servers can swing by and grab a copy. You use a Usenet reader to actually browse and download the files, sort of like how you use a torrent client to download torrents. The upside is that you don’t need to worry about whether or not a torrent is properly seeded; There’s a dedicated server that is hosting the file. The big downside here is that you actually need access to those servers. This requires a Usenet subscription, the same way you subscribe to your ISP for internet service. Not all Usenet providers have access to every server, (though many providers make attempts at parity with partnered companies.) So it’s common to need more than one Usenet subscription, for access to multiple servers for your various types of media.
Usenet subscriptions typically work in one of two ways; By usage, or by time. Usage is like a prepaid cell phone. Maybe you buy 100GB, and you can then download 100GB from the usenet servers before needing to pay again. These tend to be cheaper in the short term, but more expensive if you’re downloading tons of data constantly. Then there are the timed subscriptions, which are just like a subscription you’d expect; You pay for a month, and you have access for a month. Many people will keep a monthly subscription with their main provider, then a usage subscription with a backup. So even if their main doesn’t have a file, their backup might, and they’re only paying for the backup when their main is failed.
If this sounds similar to cloud server sites like Mega or Google Drive, that’s because it is. And it suffers from the same hurdles; Content owners can issue DMCA takedown notices on the media they own, and force the Usenet provider to remove it from their server. Usenet has historically been a more solid way to find full file downloads, but that was largely because content owners hadn’t bothered checking them for potential takedowns. In the past few years that has changed, and files often get taken down shortly after they’re posted. If a download fails on usenet, it’s often because the file got taken down while you were downloading it.