snorrwe - Migrating my GitHub repos to Forgejo

Recently I've began migrating from my GitHub profile to self-hosted Forgejo instance.

I'm not fully commited at the moment, so I figured I'd like mirror my GitHub repos.

Including private repos, I have over 200 repositories, so I really didn't feel like doing it manually.

Dependencies

You'll need a couple of tools for this.

jq and curl are probably available in your distros package manager. For gh see the official site.

Solution

First setup some environment variables

Set GH_USER to your GitHub username.
Set GITHUB_TOKEN to your GitHub auth token. This will be used for authorization when pulling private repos.
Set FORGEJO_TOKEN to your Forgejo auth token. To create a new token go to your settings page, then navigate to "Applications". You'll need repository read-write access.
Set FORGEJO_URL to your Forgejo instance base url. For example: https://forgejo.example.com

I didn't want to migrate forks, but if you do want them, just remove the second line.

1000 is the limit of repos fetched, make sure this number of high enough.

Fetch all of your repositories and store them in a json file. Technically you could just pipe the output directly into the next command, but I wanted to have a checkpoint in case something goes wrong.

gh repo list $GH_USER --json name,isFork,visibility,url -L 1000 | \
    jq -r '.[] | select(.isFork | not )' | \
    jq -s | \
    tee github-repos.json

Then, create a mirror for each repo.

Set uid to your Forgejo userid.

jq -c --raw-output0 --arg GITHUB_TOKEN "$GITHUB_TOKEN" \
'.[] | {
    clone_addr: .url,
    repo_name: .name,
    mirror: true,
    private: .visibility == "PRIVATE",
    auth_token: $GITHUB_TOKEN, 
    uid: 1
}' < github-repos.json | \
xargs -0 -I{} curl -X POST "$FORGEJO_URL/api/v1/repos/migrate" \
    -H "Authorization: Bearer $FORGEJO_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{}'

Take each of the repos and map them into a request to POST /api/v1/repos/migrate. For the documentation see $FORGEJO_URL/api/swagger.

private: .visibility == "PRIVATE" keeps the private repos private, and the public repos public. You can also hard-code true/false if you want all your repos to be public or private.

The easiest way to pass an environment variable to a jq query is to use a variable --arg.

Use null bytes as the separator jq --raw-output0 and xargs -0, using newline terminated lines messes up the quotes in the request body.

If you're feeling fancy, you could also use GNU parallel instead of xargs, but I was worried about hitting some rate limit, and it didn't take too long serially.