Stred: Streaming Tree Editor. Like sed but for JSON
git clone
Log | Files | Refs | README

commit 7e698214ef723bba55da05fbb55c359e9bf33f76
parent 0da98a3d83b5aa6ccae73658ef0692c4023bc70f
Author: Charlie Stanton <>
Date:   Wed, 26 Apr 2023 12:52:17 +0100

Adds a worked example to the README

Diffstat: | 105+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 105 insertions(+), 0 deletions(-)

diff --git a/ b/ @@ -17,6 +17,111 @@ I'm hoping it will be capable of nearly as much as jq, with comparable or better ## Usage +Below is something between a tutorial and a reference guide, but before that let's go through an example. +Suppose that we want to count how many children the JSON root node has. +For this example we'll assume that the JSON root is either an array or an object. + +First of all, our overall input will be a JSON value and our output will be a number, so it makes sense to use the `-n` flag which disables the automatic outputting of every token. +This flag is really handy when we aren't keeping most of the data the same and only making a few changes. + +Now we know we want a stred script that outputs the number of children of the root node, so we decompose the problem and think about how exactly we want to count that? +Knowing that every stred script is executed by running it on each token of the overall JSON value, the logical steps are as follows: + +* 1 - When the program starts, initialise some counter to 0 +* 2 - Each time we pass a token that is a child of the root node, increment the counter +* 3 - When the program ends, output the counter + +Starting with step 1, we decide that out counter is going to be a single numeric atom stored in the X register. +We then break down step 1 into smaller steps. + +* 1.1 - Are we at the start of execution? +* 1.2 - If we are, set the content of the X register to 0 + +Focusing on step 1.1, to test if we are at the beginning of the program, we can test for the very first `{` or `[` of the JSON input. +Those tokens have an empty path (as they are part of the root node), so we test for them with two steps. + +* 1.1.1 - Is the path empty? +* 1.1.2 - Is the value either `{` or `[`? + +We want to test that *both* of these conditions are true and if they are we will have some number of commands we want to run. + +For step 1.1.1, we want to substitute the path for itself if it is empty, so we just use an empty path substitution: `S//`. + +Whatever command comes after it will be executed only if it succeeds, so we then check the value register with a substitution that doesn't change the value register but checks that it is the start of an array or object: ``s/[`{[`]/``. + +We will want to put some commands after these (that will only be run if both succeed) so we put some curly brackets `{}` to encapsulate the commands we'll use for step 1.2 so that the whole group of them are only run at the start of the program. + +This gives us our step 1.1 commands: ``S//s/[`{[`]/{}``. + +Now for step 1.2, we want to set the X register to 0. We'll use a substitution to introduce the 0 but since we can't substitute on the X register directly, we break it down into smaller steps. + +* 1.2.1 - Delete the contents of the value register +* 1.2.2 - Set the value register to 0 +* 1.2.3 - Swap the value register with the X register + +Step 1.2.1 is easy, just a `d` command. + +Step 1.2.2 uses a substitution on the value register, this time we want to use the output `=<stuff>=` syntax to output a 0 to the value register. +We could use ``s/=`0`=/`` for this, but it's too much effort and there is a shorthand that we can use instead of ``=`0`=`` so we have `~0~` instead. +Even better, when we use `~` as the delimiter for a substitution it is still included in the subex so we can also leave out the `/`s. +This means for step 1.2.2 our command is simply `s~0~`. + +Step 1.2.3 is also easy, just a `x` command. + +This means our command for step 1.2 is `ds~0~x` and combining it with step 1.1: ``S//s/[`{[`]/{ds~0~x}``. + +Now we carry on, let's look at step 2 + +* 2.1 - Is the token a child of the root node? +* 2.2 - If it is, increment the number in the X register + +Before we break down 2.1 into commands, we need to make an observation. +If a child value is an array, then it has two tokens that are children of the root node, a start and an end. +Objects have the same problem. +We want to make sure we only count one of these, so we'll only be counting the starting tokens. + +* 2.1.1 - Does the path have a length of 1? +* 2.1.2 - Is the value something other than an array or object closer? + +For step 2.1.1, we again just use a path substitution, with `,` that represents any single JSON value (null, boolean, number or string): `S/,/`. + +For step 2.1.2, we want to check that it is either a value `,` or the beginning of an array or object so we use ``s/,|[`{[`]/``. + +Once again we will want to run all of the step 2.2 commands only if both of these checks pass, so we add some more curly brackets for the step 2.2 commands after them: ``S/,/s/,|[`{[`]/{}``. + +Breaking down step 2.2, we know that our count is in the X register but we can't substitute there so we'll need to do all the work in the value register. + +* 2.2.1 - Swap the value register and X register +* 2.2.2 - Increment the value register +* 2.2.3 - Swap the value register and X register + +Steps 2.2.1 and 2.2.3 are easy and we've seen them before: `x`. + +Step 2.2.2 is interesting as we will need a subex that increments a value. +Subexes can add the outputs of other subexes with the `+` operator. +We want to add the input to 1, so we use `%` to copy across the input and then `~1~` to output a 1. +Both of these outputs get captured by the `+` which goes after them and adds them, giving us the increment we desire: `s/%~1~+/` + +With this, we have `xs/%~1~+/x` for step 2.2 and ``S/,/s/,|[`{[`]/{xs/%~1~+/x}`` for the whole of step 2. + +Finally, we break down step 3 + +* 3.1 - Are we at the end of the JSON input? +* 3.2 - If we are, output the content of the X register + +Step 3.1 is very similar to step 1.1, so I won't break it down any further, hopefully it's clear that what we want is ``S//s/[`}]`]/{}`` so step 3.2 can go inside the curly brackets. + +Step 3.2 is very simple. +stred has a command, `p`, to output the contents of the value register so to output what's in the X register, we use `xp`. + +This means the commands for step 3 are: ``S//s/[`]}`]/{xp}``. + +Because the three steps don't overlap on which parts of the JSON input they work with, the order we write them in is actually arbitrary. +Since we developed our solution going step 1, then 2, then 3 I'll write it that way, but personally why I needed this script I found it easier to write the increment part first, so I did. +stred is very good at letting you write your commands in the order you think of them, which is handy because although writing stred scripts is manageable, reading them is painful. + +Our final script is: ``` S//s/[`{[`]/{ds~0~x} S/,/s/,|[`{[`]/{xs/%~1~+/x} S//s/[`]}`]/{xp} ``` + ### Registers and Atoms Commands operate on data that is store in 'registers'. There are 5 registers: path, value, X, Y and Z.