I want to rip the contents of a pay website, but I have to log in to their web site on a web page to get access

Does anyone have any good tools for Windows for that?

I’m guessing that any such tools must have a built in browser, or be a browser plugin for it to work.

    • zabadoh@lemmy.mlOP
      link
      fedilink
      arrow-up
      2
      ·
      edit-2
      1 year ago

      Webcopy looks promising if I can get the crawler part of it to work with this site’s authentication…

      edit: I couldn’t get Webcopy’s spider to authenticate correctly.

      Webcopy uses the deprecated version of Internet Explorer in Windows 10 as a module, and I can log into the website using the Capture Forms browser dialog, but the cookies or whatever else don’t translate over to the spider.