Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

NodeBB

  1. Home
  2. uncategorized
  3. I'm now writing a JSON parser in Koka.

I'm now writing a JSON parser in Koka.

Scheduled Pinned Locked Moved uncategorized
31 Posts 12 Posters 4 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • sun@shitposter.worldS sun@shitposter.world
    @volpeon I have a CS degree and I'm ashamed I don't know what most of this means
    ? Offline
    ? Offline
    Guest
    wrote last edited by
    #13
    @sun @volpeon the only one that i'm not super clear what they mean is 'composable'

    'composable'

    https://en.wikipedia.org/wiki/Composability

    stateless & self-contained apparently, though the wikipedia article needs love so it's probably a recent-ish term

    and maybe 'ambient state'

    'ambient state' seems pretty straightforward : just a background state

    https://www.hp.com/us-en/shop/tech-takes/what-is-ambient-computing
    1 Reply Last reply
    0
    • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

      Long term plan is an ActivityPub server which implements my ideas to fix microblogging (see my recent posts). If nobody picks them up, I'll just do it myself. Either it turns out they're bad and it was simply a fun little experiment, or they're good and maybe this prompts other servers to follow suit

      ? Offline
      ? Offline
      Guest
      wrote last edited by
      #14

      @volpeon been thinking on your thoughts/blogs/posts here and really excited to see what you come up with!

      Definitely would love to try out what you cone up with

      1 Reply Last reply
      0
      • crw@cybre.clubC crw@cybre.club

        @volpeon@icy.wyvern.rip Exactly.

        crw@cybre.clubC This user is from outside of this forum
        crw@cybre.clubC This user is from outside of this forum
        crw@cybre.club
        wrote last edited by
        #15

        @volpeon@icy.wyvern.rip BTW aber Vorsicht, mit zeitgemäßen Funktionen und einer anständigen App für iOS und Android könntest Du erfolgreicher sein als Mastodon.

        1 Reply Last reply
        0
        • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

          Afterwards I'll do JSON-LD. Straight from baby mode to "oh god oh fuck, why am I doing this to myself" ​​

          volpeon@icy.wyvern.ripV This user is from outside of this forum
          volpeon@icy.wyvern.ripV This user is from outside of this forum
          volpeon@icy.wyvern.rip
          wrote last edited by
          #16

          Since Koka has no network stdlib yet, I'll have to implement URI and IRI handling first ​​

          stefan@akko.lightnovel-dungeon.deS volpeon@icy.wyvern.ripV 3 Replies Last reply
          0
          • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

            Since Koka has no network stdlib yet, I'll have to implement URI and IRI handling first ​​

            stefan@akko.lightnovel-dungeon.deS This user is from outside of this forum
            stefan@akko.lightnovel-dungeon.deS This user is from outside of this forum
            stefan@akko.lightnovel-dungeon.de
            wrote last edited by
            #17
            @volpeon going down the next rabbit hole?
            volpeon@icy.wyvern.ripV 1 Reply Last reply
            0
            • stefan@akko.lightnovel-dungeon.deS stefan@akko.lightnovel-dungeon.de
              @volpeon going down the next rabbit hole?
              volpeon@icy.wyvern.ripV This user is from outside of this forum
              volpeon@icy.wyvern.ripV This user is from outside of this forum
              volpeon@icy.wyvern.rip
              wrote last edited by
              #18

              @stefan The grind never ends ​​

              1 Reply Last reply
              0
              • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                Since Koka has no network stdlib yet, I'll have to implement URI and IRI handling first ​​

                volpeon@icy.wyvern.ripV This user is from outside of this forum
                volpeon@icy.wyvern.ripV This user is from outside of this forum
                volpeon@icy.wyvern.rip
                wrote last edited by volpeon@icy.wyvern.rip
                #19

                Positive side effect compared to Haskell: I can unify the URI and IRI code because if there aren't any standards, I am the one making them ​:brdCool:​ In Haskell, they were in separate packages and it was kinda clunky

                1 Reply Last reply
                0
                • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                  Since Koka has no network stdlib yet, I'll have to implement URI and IRI handling first ​​

                  volpeon@icy.wyvern.ripV This user is from outside of this forum
                  volpeon@icy.wyvern.ripV This user is from outside of this forum
                  volpeon@icy.wyvern.rip
                  wrote last edited by
                  #20

                  Well, guess what
                  I need to write a module for unit testing, too
                  ​​

                  legion495@mk.absturztau.beL volpeon@icy.wyvern.ripV 2 Replies Last reply
                  0
                  • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                    Well, guess what
                    I need to write a module for unit testing, too
                    ​​

                    legion495@mk.absturztau.beL This user is from outside of this forum
                    legion495@mk.absturztau.beL This user is from outside of this forum
                    legion495@mk.absturztau.be
                    wrote last edited by
                    #21

                    @volpeon@icy.wyvern.rip An absolut unit by birb

                    1 Reply Last reply
                    0
                    • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                      Well, guess what
                      I need to write a module for unit testing, too
                      ​​

                      volpeon@icy.wyvern.ripV This user is from outside of this forum
                      volpeon@icy.wyvern.ripV This user is from outside of this forum
                      volpeon@icy.wyvern.rip
                      wrote last edited by
                      #22

                      Well, guess what
                      The community stdlib has one and it's good, so they spared me the effort
                      ​​

                      volpeon@icy.wyvern.ripV 1 Reply Last reply
                      0
                      • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                        Well, guess what
                        The community stdlib has one and it's good, so they spared me the effort
                        ​​

                        volpeon@icy.wyvern.ripV This user is from outside of this forum
                        volpeon@icy.wyvern.ripV This user is from outside of this forum
                        volpeon@icy.wyvern.rip
                        wrote last edited by volpeon@icy.wyvern.rip
                        #23

                        I was about to add unit tests for the IRI parser and found a nice JSON file with lots of test cases. I have JSON parsing, so that's good. What I don't have is a nice way to decode the generic json type to a specific type presentation, like with Aeson's FromJSON. I could still get all the test cases from the JSON file without it, but the rest of the application needs to handle JSON as well and then a decoder abstraction would be really good to have. So writing a decoder module is next ​:brdFlat:​

                        edit: Never mind again. I keep missing stuff because I keep looking for Haskell-like solutions, but Koka is simply different.

                        volpeon@icy.wyvern.ripV 1 Reply Last reply
                        0
                        • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                          I was about to add unit tests for the IRI parser and found a nice JSON file with lots of test cases. I have JSON parsing, so that's good. What I don't have is a nice way to decode the generic json type to a specific type presentation, like with Aeson's FromJSON. I could still get all the test cases from the JSON file without it, but the rest of the application needs to handle JSON as well and then a decoder abstraction would be really good to have. So writing a decoder module is next ​:brdFlat:​

                          edit: Never mind again. I keep missing stuff because I keep looking for Haskell-like solutions, but Koka is simply different.

                          volpeon@icy.wyvern.ripV This user is from outside of this forum
                          volpeon@icy.wyvern.ripV This user is from outside of this forum
                          volpeon@icy.wyvern.rip
                          wrote last edited by volpeon@icy.wyvern.rip
                          #24

                          The application crashes when the JSON is too large. I bet it's because Koka's parser combinator library makes every single step backtrackable.

                          volpeon@icy.wyvern.ripV 1 Reply Last reply
                          0
                          • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                            The application crashes when the JSON is too large. I bet it's because Koka's parser combinator library makes every single step backtrackable.

                            volpeon@icy.wyvern.ripV This user is from outside of this forum
                            volpeon@icy.wyvern.ripV This user is from outside of this forum
                            volpeon@icy.wyvern.rip
                            wrote last edited by
                            #25

                            It's actually not too difficult to avoid backtracking if you make the subparsers smarter. I just didn't do it because it also makes them less readable, but oh well.
                            So right now I parse booleans like this, for instance:
                            string("true") || string("false"), which means it tries the first parser, and on failure the second one (and therefore the consumed input must get "un-consumed").
                            The smarter solution is to pick the right branch directly so there is no "un-consuming" necessary. Like this:

                            val c = one-of("tf")
                            if c == 't' then
                              string("rue")
                            else
                              string("alse")

                            volpeon@icy.wyvern.ripV 1 Reply Last reply
                            0
                            • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                              It's actually not too difficult to avoid backtracking if you make the subparsers smarter. I just didn't do it because it also makes them less readable, but oh well.
                              So right now I parse booleans like this, for instance:
                              string("true") || string("false"), which means it tries the first parser, and on failure the second one (and therefore the consumed input must get "un-consumed").
                              The smarter solution is to pick the right branch directly so there is no "un-consuming" necessary. Like this:

                              val c = one-of("tf")
                              if c == 't' then
                                string("rue")
                              else
                                string("alse")

                              volpeon@icy.wyvern.ripV This user is from outside of this forum
                              volpeon@icy.wyvern.ripV This user is from outside of this forum
                              volpeon@icy.wyvern.rip
                              wrote last edited by
                              #26

                              I removed all backtracking and it can handle larger JSON files now, but I still need to remove a very large chunk of the tests file to avoid the crash. ​​

                              volpeon@icy.wyvern.ripV legion495@mk.absturztau.beL 2 Replies Last reply
                              0
                              • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                                I removed all backtracking and it can handle larger JSON files now, but I still need to remove a very large chunk of the tests file to avoid the crash. ​​

                                volpeon@icy.wyvern.ripV This user is from outside of this forum
                                volpeon@icy.wyvern.ripV This user is from outside of this forum
                                volpeon@icy.wyvern.rip
                                wrote last edited by
                                #27

                                Looks like it's a known issue with ctl effects (which parsers use): github.com/koka-lang/koka/issues/279
                                There's one more idea I want to try, but otherwise I'll just accept it for now

                                volpeon@icy.wyvern.ripV 1 Reply Last reply
                                0
                                • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                                  I removed all backtracking and it can handle larger JSON files now, but I still need to remove a very large chunk of the tests file to avoid the crash. ​​

                                  legion495@mk.absturztau.beL This user is from outside of this forum
                                  legion495@mk.absturztau.beL This user is from outside of this forum
                                  legion495@mk.absturztau.be
                                  wrote last edited by
                                  #28

                                  @volpeon@icy.wyvern.rip How large exactly? ​​

                                  1 Reply Last reply
                                  0
                                  • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                                    Looks like it's a known issue with ctl effects (which parsers use): github.com/koka-lang/koka/issues/279
                                    There's one more idea I want to try, but otherwise I'll just accept it for now

                                    volpeon@icy.wyvern.ripV This user is from outside of this forum
                                    volpeon@icy.wyvern.ripV This user is from outside of this forum
                                    volpeon@icy.wyvern.rip
                                    wrote last edited by
                                    #29

                                    IRI parser works ​​
                                    Only the IRI resolution algorithm left, and then I can finally get to JSON-LD

                                    johann150@genau.qwertqwefsday.euJ volpeon@icy.wyvern.ripV 2 Replies Last reply
                                    0
                                    • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                                      IRI parser works ​​
                                      Only the IRI resolution algorithm left, and then I can finally get to JSON-LD

                                      johann150@genau.qwertqwefsday.euJ This user is from outside of this forum
                                      johann150@genau.qwertqwefsday.euJ This user is from outside of this forum
                                      johann150@genau.qwertqwefsday.eu
                                      wrote last edited by
                                      #30

                                      @volpeon@icy.wyvern.rip oh no, gonna loose volpeon to JSON-LD again

                                      will it be like the last time you tried that?(i.e. we don't hear from you at all for a week or so)
                                      ​​

                                      1 Reply Last reply
                                      0
                                      • volpeon@icy.wyvern.ripV volpeon@icy.wyvern.rip

                                        IRI parser works ​​
                                        Only the IRI resolution algorithm left, and then I can finally get to JSON-LD

                                        volpeon@icy.wyvern.ripV This user is from outside of this forum
                                        volpeon@icy.wyvern.ripV This user is from outside of this forum
                                        volpeon@icy.wyvern.rip
                                        wrote last edited by
                                        #31

                                        I revisited the JSON parser and wrote my own parsing library. It's not too different from the one in the stdlib, but it handles some cases more efficiently and that makes a huge difference.
                                        The old parser caused a stack overflow with a 144kb JSON file. The new parser can handle a 1mb JSON test file, but still causes a stack overflow with a 5mb file.

                                        1 Reply Last reply
                                        0
                                        Reply
                                        • Reply as topic
                                        Log in to reply
                                        • Oldest to Newest
                                        • Newest to Oldest
                                        • Most Votes


                                        • Login

                                        • Login or register to search.
                                        Powered by NodeBB Contributors
                                        • First post
                                          Last post
                                        0
                                        • Categories
                                        • Recent
                                        • Tags
                                        • Popular
                                        • World
                                        • Users
                                        • Groups