You know when you have the developer tools open in Chrome or Firefox (and IE? Who knows..) and set to the network tab, you can right-click a request and select “copy as cUrl”, and you get curl command line with a pile of flags that will reissue the exact same request? Well, if you didn’t, you can – start using it, it’s awesome for testing stuff that’s not so complex that you have to write a selenium or phantomJS script. Handy for working out how the back end (yours or someone elses..) deals with requests that can’t occur in the browser, but which can occur when some brat decides to fire up burpsuite and start sending it broken Unicode just to see if you did your homework.
But sometimes you need to write something a *little* interactive, but not much so that you need a headless or actual browser for it (or for something performance critical enough that you don’t want one) and working on something else I realized I spend an awful lot of time rewriting these curl command lines into python. Since that’s really just a straight “madlibs” insertion formula, there should really be something automatic that does that. So now there is:
So.. there you go, if you, too, do that sometimes this will do that for you. The output uses urllib3 (might add on options to use requests or mechanize instead sometime) and is just a single function called getpage that fetches it and prints the headers and the response, and a “if __name__==” snipplet to run it if it’s executed directly (yes, I’ve taken a slight liberty in pretending there’s a -i on the curl command, because really there should be). If you give it and argument, it’ll read that file for the curl command, if there’s no options it’ll use the /dev/clipboard (i.e. the clipboard on linux/osx/cygwin, nothing particularly useful on winows). There’s no filters so it’ll add everything, including headers like “Host:” and stuff that technically shouldn’t be there, but when considering filtering I decided it’s probably better to retain the integrity of the curl command and you can delete or edit them yourself (just as the with curl command).
[EDIT] ..and in less than 24h it’s gotten a bug fix – the normal way of passing headers mixes up their order, which is slightly different than curl, and matters in certain edge cases. Also added the ability to skip -I if it’s been added. Still pretty strict on input order and not much error check..