Guide Tools (Updated) Guide to downloading with gallery-dl

NikoCast6

Lurker
Joined
Dec 25, 2024
Posts
106
Reaction score
1
Originally posted in this thread where few would ever see it. I thought I'd post an update in my profile, but there's a pretty short length limit on profile posts. So here let me welcome you to my:

:HACKERMANS: _______GUIDE TO GALLERY-DL______ :HACKERMANS:
[ free + open source ]

Q: why tf should I use gallery-dl?A:it magically downloads photo albums - social media - file hosts - image hosts

Q: what sites can it download from?A:instagram + twitter + tiktok + reddit + pinterest + imagefap + redgifs + goonbox

Q: is that all?A: fuck no. it also downloads from:bunkr + gofile + pixeldrain + turbo + tube sites + a shitload more

:HACKERMANS: WHAT YOU NEED TO KNOW: :HACKERMANS:




  • The answer to whatever you want to know is on the codeberg page: https://codeberg.org/mikf/gallery-dl
    • this includes supported sites, errors (see Issues), how to install, etc.
    • this was formerly on github (https://github.com/mikf/gallery-dl), which can still be helpful for past issues/discussions
  • Benefits of downloading with gallery-dl
    • it simplifies downloading from any site it supports
    • supported sites include instagram, twitter, tiktok, reddit, imagefap, redgifs, cxxmer/kemono, pictoa, pinterest, goonbox, bunkr, gofile, pixeldrain, turbo, many tube sites, many image hosts, and a ton more
    • it makes downloading photo albums painless (especially for sites not equipped to handle image dumps. read: bunkr!)
    • it maintains a database of completed downloads so you don't unwittingly re-dl files you already saved. This also means you can return to that [instagram/reddit/whatever] profile from time to time and it will dl only the new files since last time
  • It's really fucking easy to use
    • basic usage:
      • to download a bunkr album,
        Code:
        gallery-dl https://bunkr.cr/a/RE4jsE8Q
        <- test album. try it out!
      • to download a single file from bunkr,
        Code:
        gallery-dl https://bunkr.cr/f/zGPPQCux9yOYf
      • to download all posts from an instagram profile,
        Code:
        gallery-dl https://www.instagram.com/rachelrileyrr
      • to download tagged posts,
        Code:
        gallery-dl https://www.instagram.com/rachelrileyrr/tagged
        • or replace 'tagged' with 'reels', 'stories', etc. makes sense?
        • alternatively, use
          Code:
          gallery-dl -o include=posts,tagged,reels https://www.instagram.com/rachelrileyrr
    • it's also massively configurable via config file once you learn the ropes (e.g. automatically name files however you want! organize download locations however you want! damn it's so good!)
  • You should install gallery-dl (and any command line tools) using a package manager
    • a package manager keeps all your command line software up to date, including its dependencies, and organizes them all on your system where they can find each other
      • if you're new to the command line, making sure you have needed dependencies, keeping them properly updated, and figuring out how to make the (god damn!) things find each other is the vast majority of headaches you suffer to get any (god. damned.) program working
      • these problems become moot when you use a package manager
      • srsly, especially if you're new to this, don't fuck around with downloading binaries yourself, let the package manager do it for you. And don't come bitching when you can't get it to work because you ignored this advice
    • mac: gallery-dl is available through Homebrew (most popular) & macports
      • macports has better compatibility with more systems, as Homebrew limits support to newer devices/OSes
      • example usage - it is easy:
        • Code:
          sudo port install gallery-dl
          installs the program and its required dependencies
        • Code:
          sudo port selfupdate
          keeps macports up to date and downloads latest info about available programs
        • Code:
          sudo port upgrade outdated
          upgrades all your installed programs to the latest versions
    • windows: supported pkg managers for windows are Chocolatey and Scoop. idk I don't use windows, but same concept
    • linux: linux users already know what package manager they use. shut up
    • super mega bonus tip: if everything is working and you feel comfortable, you can begin updating on your own to the nightly (aka development) version. it can take a few days or more for additions and bug fixes to make it to a release, but you're guaranteed the latest when you install the nightly. follow the directions under "Install the latest dev version directly from master" -> https://codeberg.org/mikf/gallery-dl#pip <-
  • If something used to work and now it doesn't, upgrade your software to latest versions
    • stuff breaks all the time. websites make a small change that breaks tools and developers need to make changes to keep everything pointed in the right direction. this is how stuff works
    • even if you just upgraded yesterday, check again today. devs of popular software release new versions frequently
    • if you want fixes before they're even released, check the super mega bonus tip above
  • If something is not working, look to the codeberg issues page
    • i'll even link to it for you: https://codeberg.org/mikf/gallery-dl/issues
    • don't ask about the latest bug here or anywhere else other than the gallery-dl repo - nobody knows or cares. ask about it at the link i gave 1 line above this (search for it first because somebody has probably brought it up already)
    • new bugs and stuff will be in "Open" issues. everything anyone's ever asked about will be in "Closed" issues. make sure you are checking both
  • Go forth and download
    • if you're new to the command line, remember that "new" is not the same as "hard". at one time you didn't know how to pick your nose
    • Code:
      Ctrl+C
      exits you out of running programs
    • remember to go outside every few days, now that you have this power it can be hard to stop using it

ADDITIONAL Q&A:

Q: are you affiliated with gallery-dl?A:not at all. it's just life-changing software that i've spent a lot of time learning and using, and more people should know about and use it.

Q: now that i'm using gallery-dl, will you also come over and jerk me off?A:if you're reasonably fit, young, and don't have a beard, yeah i'll probably do it.
 
i love gallery-dl but when ive used it ive gotten my accounts locked annoyingly, they think im a bot but im just an efficient collector of delightful treasures :peepoShy:. its not exclusive to gallery-dl since ive tried other similar tools and manage to trigger lockouts with those too. i just want to warn anyone following this helpful guide that when you walk the path of the scraper, you may find your accounts to be treated like those damn CLANKERS and get B& :BongoBan:, so scrape with discretion!!
 
For anyone worried, if you only read one line from this post: this is not a common occurrence, it is avoidable, and if you don't intentionally login to an account it's not even possible.

Something to be aware of if you're scraping mass amounts of content. I almost never use accounts for any site I download from - I regularly download from Instagram and TikTok, but I think the only time I have used authenticated cookies is to access age-restricted posts on Twitter. And of course you'll know if you're doing that, and can check the repos for information first. The developer is very active and extractors are designed to avoid issues, but it's also up to us to read the docs first to make sure we're using tools safely.

Might be helpful if you briefly describe what happened, when it was, and what sites you were downloading from - of course this can happen, but it is rare, still much rarer if you're not downloading on a large scale, and impossible if you don't authenticate. My goal here was to help people see how easy it can be, and what I don't want to do is scare people away because they're afraid of something that is very unlikely to happen.
 
yeah so let's talk about using authenticated cookies for twitter downloading. where on the repos should we be checking and what docs should we be reading to reduce the chance of lockouts/suspensions? rare as they may be, they do happen to some of us...
 
For sure, I don't doubt you. Btw my post wasn't meant as a challenge, more that I don't want anyone to run away in fear from a tool that has been a life changer for me in terms of efficiency, organization, and eliminating headaches.

I don't download a ton from Twitter - typically individual posts, but I have downloaded a full account or two. Those would have been in the last 6 months, and I haven't had any issue.

The best place to keep apprised of recent developments (and past issues) is the issue tracker in the gallery-dl repo. Actually there are 2 now since the move to codeberg as the issue tracker on github is still open and active. There is also a Discussions section on the github repo where people ask general questions.

Basic usage is easy, but once you learn the ropes the real power comes in building your config.json to tell your extractors how to behave. That can have a bit of a learning curve as you learn what is available and how the syntax works but over time the pieces come together and it's so nice. Many of the options are for what to download, how to name files, and where to put what, but also many options tell the extractors how to interact with the site. Extractors are custom built for each site with their own available options and exploring what those options do can make your life better/easier.

The docs are in the folder called, well, "docs". gallery-dl-example.conf shows the basic structure of a config file. configuration.rst is a behemoth of a file, which lists every option available and what it does. If you're doing extensive scraping, you'll want to spend significant time with this file. I come back to it a lot. This is where it can take a while to build your knowledge as it's exhaustive, but not written as a set of instructions. But it starts to make sense as you learn how everything goes together. Finally, gallery-dl.conf is simply a list of every single setting and its default value, in exactly the format it is used in your config file. This is always the most up to date, it gets updated as new options/sites are added.

At some point I'd like to add here a basic how-to for your config file with examples. There was a thread last month about downloading posts/threads from simpcity using gallery-dl and I made a post there where I explained how that can get out of hand fast so it's best to define a more specific target than "dl everything in this thread". The thread got deleted anyway as they didn't want to encourage scraping the forum. If a mod reads this and wants to pm me my post that would be cool, and I will remove any instruction about scraping threads other than to explain why that's a bad idea anyway.

For everyone else feel free to post questions in this thread, I'm happy to share what I have learned.
 
Gallery dl has been a godsend for me for sure since i started using it earlier this year.

just this past week i looked into setting up a automatic tiktok plus instagram story downloader and found a git project that would use gallery dl for instagram and yt dlp for tiktok. after altering the project a lot i am finally able to have an automatic monitor for tiktok stories and posts, with automatic downloads and also follow up checks to replace posts with higher quality ones after certain time checks, as tiktok at least doesn't usually give you the best resolution at all times especially right after publishing.

I still need to add in automatic recording and downloading of tiktok lives and then move onto the instagram portion as well, to capture automatically primarily stories there as well using the same setup, but it looks pretty nice so far.

sorry if this is the wrong place to post, but since it is a gallery dl thread i thought this could be somewhat relevant or helpful for people looking into it and options using it.
 
Not having much luck with anything unfortunately. :Sadge:

Trying everything below with VPN, Instaloader instantly 429s. I see an issue saying the profile API endpoint they use is deprecated, someone has a pull request with a fix so maybe I'll try it again if that gets merged.

A tutorial online says to try gallery-dl with the simulate option first, but it says profile not found. One issue said to rip NSFW accounts, even public ones, you probably need to log in.

Tried making a new account for gallery-dl or something like Gramoire, but it seems IG shadow blocks Protonmail emails, since it won't take my verification code. Tried again with a 10 minute mail account and it took the code, but then they wanted a phone number. Don't want to try with one of my real accounts / risk linking a throwaway to my real accounts.

---

Are you guys able to download IG posts without logging into/providing cookies to whatever tool you're using? Or are you logging in and risking a real account, or just were able to make throwaways you don't care about? The main account I want to rip is not private so I was hoping I'd be able to do something without logging in.
 
The functionality for ig is just dead until the owner updates it and they probably won't update it for a while
 
Back
Top