Editing RakuDoc, CRO'ing it to Github

All I want is to correct a one-letter typo The ask, expressed by our website users, is simple enough, edit a source from a GitHub repo in a browser, then save the edited version back. How difficult is that? It's easy to point the browser at the file in the repo, and GitHub has an editor ... The problem comes from authorisation, and bad actors wanting to add stupid stuff to files - like: I'm the brilliantish shop for high-ent wazzoo's. Come take a look at http://rip-me-off.darkweb.cosmo Github's editor is easy to use - if you have commit permissions on the repo, otherwise: you need to clone the repo, correct the one letter typo in a single file, raise a PR. Which is fine if you know what a 'clone' is, or a 'repo' is, or a 'PR'. But its only one letter!?? Also, our website text source is written in RakuDoc (not MarkDown, but that's another story) and so it would be nice to be able to see what the change looks like. As always a seemingly trivial mistake, such as deleting a

Apr 14, 2025 - 22:11

All I want is to correct a one-letter typo

The ask, expressed by our website users, is simple enough, edit a source from a GitHub repo in a browser, then save the edited version back.

How difficult is that? It's easy to point the browser at the file in the repo, and GitHub has an editor ...

The problem comes from authorisation, and bad actors wanting to add stupid stuff to files - like: I'm the brilliantish shop for high-ent wazzoo's. Come take a look at http://rip-me-off.darkweb.cosmo

Github's editor is easy to use - if you have commit permissions on the repo, otherwise:

you need to clone the repo,
correct the one letter typo in a single file,
raise a PR. Which is fine if you know what a 'clone' is, or a 'repo' is, or a 'PR'.

But its only one letter!??

Also, our website text source is written in RakuDoc (not MarkDown, but that's another story) and so it would be nice to be able to see what the change looks like. As always a seemingly trivial mistake, such as deleting a < in the wrong place, can have severe effects.

So, it would be nice to see an HTML rendering of the edited text before submitting it back.

The first steps into the forest

It seems doable:

Get the content of the file from Github - there is a simple URL and the Javascript fetch function.
Put the content into an online editor - there are a few editor libraries, which give you all sorts of control; I chose one.
Get the content back from the editor at some point, send it to a renderer - I put my Rakuast::RakuDoc::Render into a docker container, wrote a very simple Cro app that has all the logic for a websocket. The app expects RakuDoc source, and sends back the rendered HTML, which is then added into a div next to the online editor.
Send the edited version of the file back to Github - oh. I'm lost in the forest.

I'll not explain tasks 1-3 above and focus on task 4.

The path through the forest

While getting the content is easy, returning it is hard. The API documentation is written for developers who understand Github and HTTP requests.

I spent two days wandering though the API documentation, searching for tutorials, (and Github's own discussion forum seems to be flooded with trolls, so is not a good place to look).

I even resorted - gasp - to ChatGPT. Whilst the code it suggested was not remotely what I wanted, the 'solutions' demonstrated a remarkably simple truth. Perhaps I should have realised it on my own, but in all my documentation searches I had not read a hint about it.

The simple truth: getting a patch of a file to the repo maintainers could only be done in a sequence of steps, not the single step that I was looking for.

It is obvious that Github wants to ensure that only authorised users can change the state of a repo. Authorisation is accomplished by accompanying any request for a change with a token that has a limited duration, is unique to a recognised user on Github, and has permissions for the repo.

While it is possible to get and use such a token for an arbitrary user, by sending them to Github to log in, the process is an added layer of complexity. For simplicity, we shall consider that a token has been generated and call it id-token. (For development, I used one generated for myself)

The steps on the map

In generalised terms, here are the steps to sending a suggestion for editing a file to the Github repo. (I'll explain some of the jargon with each step).

Obtain the contents of the file and its sha from repo.
- Github stores all the information developers use in files. Each is tagged with a sha, and is located in a repository or repo.
- Actually Github has a two layer structure of owner/repo-name, so in this article, when I say repo, I mean the combination of owner/repo-name. For example, the site I am working on uses the source from the Raku/doc Github storage, where Raku is the organisation that owns the storage, and doc is the repo in which Raku holds its documentation suite.
- The sha is the encrypted sum of the entire file, and is presented as a 40 digit hexadecimal string. The maths of shas is quite interesting but not relevant here. Suffice it to say that if a file is changed at all, the shas of the two files (before the edit and after it) will be different (for the pedantic, I'll add 'with high probability').
- In fact, this step can all be done in the browser because there is no change in state of the Github storage.
Obtain the latest sha or commit for the repo.
- Just as the contents of one file at one point in time can be identified by a sha of the file, so too can the contents of the whole repo.
Create a new branch or reference point in the repo.
- In order to retain both the existing content of a repo and the suggested new content, the new content is held in a new part of the repo, and this new 'section' is called a branch. It can also be thought of as a reference point in the history of the repo. At some point the maintainer of the repo can accept the new content (merging it into the main part of the repo), or reject it.
- Creating a branch changes the state of the repo, so Github only allows a recognised agent to do this. So, this step has to be accompanied by an id-token.
- The information needed by Github for this step is
  1. The name of the repo
  2. The sha of the repo
  3. An id-token
  4. A name for the branch
  5. A description of the branch
Move a copy in BASE64 format of the edited file to the new branch
- BASE64 is a way of coding a file so that it can be transmitted safely across the internet.
- This step needs
  1. The name of the repo
  2. The name of the branch
  3. The path of the file inside the branch
  4. The sha of the original file
  5. The new contents of the file in BASE64 format
  6. An id-token
Raise a PR for the branch
- A PR is a pull request, and it is a suggestion to the repo maintainer to merge the suggested changes of the file into the main content of the repo.
- This is the end result we want.
- A branch may contain changes to many files (not just one), and so a PR is for the whole branch (for the pedantic: 'typically').
- This step needs
  1. An id-token
  2. The repo name
  3. The branch name
  4. A title for the PR
  5. A description of the PR

Even though only one letter in one file may need to be changed, the API is set up (reasonably) for many changes to many files.

Beware ... the trolls and bogey men

Can we do this in the browser? Uh, well ... yes, but No.

The problem is that everything in a browser can be examined by the world. So if you put an id-token in a browser, it can be extracted from the browser, and then a bad actor can use the token to do silly stuff.

So, each of the steps above which require an id-token have to be executed from a place that can be defended, such as inside a container where the id-token can be hidden and renewed on a regular basis.

Creating a hallowed glade in the forest

The solution is to create a safe location - hallowed glade - in which the id-token can be stored, and from which the calls to Github can safely be executed.

So let us create a container with a Cro application. The container will have a websocket to accept the edited file from the browser, and then separately interact with Github.

The Cro documentation was written by a genius for developers with experience, unlike me. They are daunting and confusing - it took days of work to figure things out. However, Cro does make it easy to set up the steps listed above.

The first thing to realise is that Cro::HTTP::Client and Cro::HTTP::Server / Cro::HTTP::Router are - at least externally - independent of each other, but we will need both.

We will need the Server to listen for the data coming down the websocket, and then Client to move the edited file to the Github repo.

Since step 1 above is done in-browser, it needs to be implemented in Javascript. I will not deal with that here, except to say, that when the content is fetched from Github, the sha of the file is saved. The in-browser code finally sends along a websocket a JSON object with the following fields:

repo - the owner/repo-name combination
path - the path of the file inside the repo
sha - the sha of the file
content - the edited content in BASE64 form
editor - the person editing the file
comment - why the edits are made
patch - a Patch between the original and edited files

The last three are not strictly needed, but can be used for some sanity testing.

Lets assume the web socket has a route suggestion_box, then the following is the code for a Cro application to get the data

use Cro::HTTP::Log::File;
use Cro::HTTP::Server;
use Cro::HTTP::Router;
use Cro::HTTP::Client;
use Cro::HTTP::Router::WebSocket;
use DateTime::strftime; # a helper to provide nice time formats

my $patch-limit = 5 * 2 ** 10; # 5k limit of chars
my $comment-limit = 1 * 2 ** 10; # 1k limit of chars
my $pr-update-duration = 10 * 60 * 60; # every ten minutes

my $host = '0.0.0.0';
my $port = 60005;
my @suggestions;
my $busy = False;

my Cro::Service $http = Cro::HTTP::Server.new(
    http => <1.1>,
    :$host,
    :$port,
    application => routes(),
    after => [
        Cro::HTTP::Log::File.new(logs => $*OUT, errors => $*ERR)
    ]
    );
say strftime(DateTime.now, '%v %R') ~ ': starting up.';

$http.start; # this starts the server part of the application

# the following is an asynchronous structure which listens for events
# then does the block for that event. The main event is to stop 
# the app when a control signal is raised.
react {
    whenever signal(SIGINT) {
        $http.stop;
        say strftime(DateTime.now, '%v %R') ~ ': closing down.';
        done;
    }
    whenever Supply.interval($pr-update-duration) {
        # every pr-update-duration seconds, this block is run
        # it calls the code for raising a PR
        unless $busy {
            $busy = True;
            raise-pr( @suggestions.pop ) while @suggestions;
            $busy = False;
        }
    }
}
# these routes are what define the application
sub routes() {
    route {
        get -> 'suggestion_box' { # the route for the websocket
            web-socket :json, -> $incoming {
                supply whenever $incoming -> $message {
                    my %json = await $message.body;
                    if %json { # no sha, no play
                        my $response = sanitise( $json );
                            # place limitations on the incoming data
                            # add 'cancel' to ignore data
                        @suggestions.push: $json.hash.clone
                                unless $json;
                            # store the data to be sent as PR
                        emit({
                            :timestamp( strftime(DateTime.now, '%v %R')),
                            :$response
                        })
                            # send back to the websocket JSON with
                            # a time stamp and a reponse code
                    }
                    elsif $json {
                        # send back a handshake when websocket opens
                        emit({ :connection })
                    }
                }
            }
        }
    }
}
sub sanitise( %edit --> Str ) {
    my $response = 'OK';
    for %edit.keys {
        when 'editor' { 
            %edit .= subst(/\W/, '_', :g)
                          .substr( ^21 ) 
        }
        when 'patch' {
            if %edit.chars > $patch-limit {
                %edit = True;
                $response = 'TooManyChanges'
            }
        }
        when 'comment' { %edit .= substr(^$comment-limit) }
        when .any  {}
        default       { %edit{ $_ }:delete } 
                   # remove unwanted fields
    }
    $response
}
sub raise-pr( %an-edit ) { ... }

The Server code handles all the interaction between the browser and the 'Hallowed Circle'.

Now we execute the remaining steps ( 2 - 5 ) of the Github sequence.

sub raise-pr( %edit ) {
    #| Define the secret token. When the container is run,
    #| the token can be passed as an environment parameter. 
    #| Later, we can refactor to obtain it to identify the
    #| user making the edits.
    my $id-token = %*ENV;
    my $base = 'https://api.github.com/repos';
    # create a descriptive branch name
    my $branch = strftime(DateTime.now, '%v')
            ~ "_{ %edit }";
    # get the repo information
    my $resp = await Cro::HTTP::Client.get(
        "/{%edit}/git/ref/heads/main"
    );
    my %data = await $resp.body;
    my $commit = %data