How to get parameters for any RTMP{E}{T}{S} stream

(1/2) > >>

Guitaroomasta:
First a little backstory:

I "stumbled" upon this method of getting the information needed for RTMP streams over labor day weekend when it dawned on me that I could possibly use use "rtmpdump" (an external program that records all kinds RTMP streams) to dump streams from sites like TLC, TBS, Crunchyroll, etc. to the Hard Drive and then to PlayOn since PlayOn's native RTMP capabilities are limited.

To make rtmpdump work I needed the correct parameters; these parameters can be obtained using another program included with rtmpdump called rtmpsuck... But unfortunately rtmpsuck only works using Linux (unless you have two computers, then you can use it on Windows) Over what ended up being a four day weekend (Yay no classes! Thanks for the drizzle Hurricane Earl :roll:) I tried in vain to figure out how to get rtmpsuck to work. (I'm not a Linux guy)  When I finally found some decent steps on how to set it up. SUCESS! but the success was short lived as soon found out that I couldn't get the PlayOn server to launch the external program (rtmpdump) when necessary.  So I just about gave up.

But soon after I gave up (like 5 mins later) I noticed while looking at the custom RTMP XML string used in another script, that the tags used in the XML where identical in spelling and casing (capitalization) to those used by rtmpdump... hmmm... interesting.  So I figured maybe, just maybe if I add my own "custom" tags that match the paramaters used natively by rtmpdump that some streams that didn't work before would start to work, and... IT WORKED!  Crunchyroll which did not work before because an improper implementation of some parameters and an absence of some other necessary parameters "app" and "flashVer" was now working.  Since then I decided to write this guide to show other devs how to get all the RTMP paramters that your browser sends when you watch videos from certain sites.  Take note, the process involves virtualization so your going to need a relatively beefy PC.

Unfortunately I end this little story with good and bad news.  I regret to tell you all that it still doesn't work with all RTMP{E}{T}{S} streams because PlayOn doesn't seem to send swf verification ("swfVfy" in rtmpdump) when the tag <swfVfy> is added. This is necessary for many but not all RTMPE streams. The good news is that I have FINALLY figured out how to get PlayOn to launch rtmpdump with the proper commands (PlayOn needs be running in User Mode among other things) and to read the file that rtmpdump creates.   So this guide may see an update in the future.  

Now enough with the pontificating and onto the guide.

What You'll Need:
[*]A Dual Core Windows PC (should work on XP, Vista, or 7)
[*]3GBs or more disk space
[*]3GBs of RAM (As much as possible)
[*]Virtualbox (Download it
[Link Removed] You are not allowed to view links. Please login or register
Register or Loginhere
)
[*]My pre-prepared, self-extracting,Linux Mint virtual machine (Download it
[Link Removed] You are not allowed to view links. Please login or register
Register or Loginhere,
1GB Sorry! archive password is: playonrtmp)
[*]rtmpdump for Windows (Download it
[Link Removed] You are not allowed to view links. Please login or register
Register or Loginhere
)
[*]Optional:
[Link Removed] You are not allowed to view links. Please login or register
Register or LoginStreamTransport
(A free streaming video downloader, like Replay Media Catcher but better)
[/list]

What you get:
[*]A fully prepared Linux Mint install, with unnecessary packages (software like OpenOffice) removed. (And the disk image is still almost 3GBs)
[*]VirtualBox Guest Additions preinstalled so you should be able to use Full-Screen and Seamless Modes on first boot.  You will also be able to copy and paste to and from the virtual machine.  Over all the experience should be smoother (and less taxing on the host PC) with the Guest Additions installed.  If they don't work Guest Additions can be re-installed from the "Devices" menu of the VirtualBox VM window.
[*]rtmpdump packages preinstalled.
[*]Firefox customized with Ad Block Plus (essential) and HTTPFox, an http analyzer similar to fiddler2, but nowhere near as capable.
[*]Charles Web Debugging Proxy, a shareware http analyzer similar to fiddler2 in capability but more confusing to use.
[/list]

Passwords for Linux Users
vbox (Default User): "virtual"
rtmp: "rtmpdump"


Procedure:

Step 1: Download the programs listed above
Step 2: Install VirtualBox but not the Host-Only adapter.  You won't need it for this.
Step 3: Extract Guitaroomasta_LinuxMint.exe to a directory of your choice, or use the default which should extract to a folder VirtualBox created during setup.
Step 4: Start VirtualBox and click "File" --> "Import Appliance"


[Link Removed] You are not allowed to view links. Please login or register
Register or Login


Follow the prompts to import the virtual machine I created.


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 5: Start the machine and wait for it to boot up.


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 6: Once the machine has finished booting.  Put it into either Full-Screen Mode (Right Ctrl+F) or Seamless Mode (Right Ctrl+L)


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 7: Open a terminal window by clicking the icon on the quick-launch panel or from the application menu


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 8: In the terminal type (or cut and paste) "sudo su rtmp" and type the admin password: "virtual" (This changes the terminal user from the logged in admin "vbox" to the second user "rtmp")


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 9: Next type "rtmpsuck" in the terminal window.


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 10: Now open Firefox (Application Menu --> Internet --> Firefox) and go to a website that you know uses some form of RTMP (I'll use Crunchyroll in the example)


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 11: When the videos page finishes loading switch back to the terminal window.  You should see all the RTMP parameters the flash player sent in order play the video.  NOTE: Sometimes rtmpsuck will fail to successfully start saving a file, if possible try a different video from the same site to ensure that the failure wasn't a fluke due to a cached page or something else.


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 12: While in the terminal window, press Ctrl+C twice to force rtmpsuck to stop.  The video in the browser will stop streaming.  (rtmpsuck works by acting as a transparent proxy that monitors RTMP traffic to get parameters that are sent to play a video)


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 13: At this point I recommend you copy and paste the parameters from the terminal into a text file in Windows to save for later so you have and example of how your particular site works.

Step 14: If you are in Full-Screen mode, minimize the virtual machine. Extract rtmpdump for Windows to a folder of your choosing. (I suggest the desktop for the time being)

Step 15: Open a Command-Prompt window and change directories to the folder where you extracted rtmpdump. (Example: cd Desktop\rtmpdump*)


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 16: Now in this step you will use rtmpdump to double-check all the information you got when using the virtual machine, by attempting to download the video in a seperate environment. Parameters can be givin in any order. A brief parameter key:
-r  corresponds to "tcURL" :The RTMP url in the form rtmp{e}{t}{s}://[hostname][port]/[app]
-a corresponds to "app"     :The application for the stream
-y corresponds to "Playpath":The actual video file on the RTMP server
-s corresponds to "swfUrl" :The flash players url
-W corresponds to "swfVfy": Calculates a SHA256 hash from the swf to send to the RTMP server for verification.  Deprecates swfUrl when used.  (swfVfy doesn't work when used in PlayOn from what I can tell)
-f corresponds to "flashVer": The version of flash used to play the stream (LNX for Linux, WIN for Windows)
-p corresponds to "pageUrl": The url of the page that called the stream
-o corresponds to "Saving as":This can be whatever you want it to be. You must include it when dumping an RTMP stream other wise the file will be written to stdout (the Command Prompt window/terminal)
-V                                    : Dumps the file in verbose mode, this allows you to see exactly what rtmpdump is doing while making (or attempting to make) a connection to the server.


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



[Link Removed] You are not allowed to view links. Please login or register
Register or Login



If rtmpdump starts successfully GREAT!  If not run the command again with the verbose parameter (-V) and see if rtmpdump gives you a reason as to why it didn't connect.

NOTE: rtmpdump will tell you when the server request swf verifcation, (-W) it will attempt to continue without out it.  Even if it starts to dump it will eventually fail if the swfVfy (-W) parameter was not supplied.  
[Link Removed] You are not allowed to view links. Please login or register
Register or LoginFull rtmpdump documentation is available here.



Step 17: To stop rtmpdump from err dumping... use Ctrl+C (you may have to press it twice, NOTE: Ctrl+C will end any process running at a command prompt/terminal)


[Link Removed] You are not allowed to view links. Please login or register
Register or Login



Step 18: Now we're comming into the home stretch, try using the information you have learned to get the RTMP stream working in PlayOn using a string this format.

Code:

local rtmpXML = tcURL..|<rtmpMedia version="1.0" super="1"><mediaPath>'..playPath..'</mediaPath><connectParams><swfUrl>'..swfUrl..'</swfUrl><pageUrl>'..pageUrl..'</pageUrl><app>'..app..'</app><flashVer>'..flashVer..'</flashVer></connectParams></rtmpMedia>'

These are the only parameters that I have gotten to work.  Try others and see if they work.

Step 19: Now when you want to do another site simply run rtmpsuck in the virtual machine terminal again.  To go back to the "vbox" user type "su vbox" (password for the "rtmp" user is: "rtmpdump") or you can just close the current terminal and and open another terminal.  Simply shutdown the virtual machine when you are done with everything by opening the application menu and choosing "Quit."



That's it!  I hope you find this guide useful in your future script/plugin writing.  Best of luck!

hotTamale:
Guitar,

Wow, amazing work.  RTMP has always been the bain of plugins/scripts.  Just getting the Virtual Box up and running with rtmpsuck had to be a task.  When I find some free time I'll be checking this out.

Thanks for the step by step,
hotTamale

crawl9992:
Thanks for this posting Guitar. I think there should be more users thanking you for this info :)

Since I also don't do anything on Linux, I was thinking of a way to try out your steps with rtmpdump without having to setup the VM.  After a bit of searching, I found a list of step by step instructions at
[Link Removed] You are not allowed to view links. Please login or register
Register or Loginhttp://stream-recorder.com/forum/correctly-use-rtmpdump-rtmpsrv-t6906.html
.

The steps describe how you can use rtmpsrv instead of rtmpsuck to get the resulting rtmpdump command line.  One nice thing about using rtmpsrv is you don't need a separate VM so you can skip the Linux part ;).  I tried the steps out and I was able to successfully get an rtmpdump command line for the TNT and A&E sites properly.

The only catch is you need to know the actual server where the rtmp file is located ahead of time.  However, I found that you can find this by using a traffic sniffer like WireShark (when you know what you are looking for).

Anon123:
Is there any other chance to get your pre-prepared, self-extracting,Linux Mint virtual machine because on Mega upload you have to be premium member to get files larger than 1GB.

Thanks for reply

famine:
You can get linux mint from here.


[Link Removed] You are not allowed to view links. Please login or register
Register or Loginhttp://www.linuxmint.com/download.php

Navigation

[0] Message Index

[#] Next page