Parsing a JSON file



  • Hi,
    It is possible for you guys to give a tutorial on how we read a json file from a http request or as a file from SD ? I am quite new into C, and I have tried to figure out how the weather app is working, but I am clueless :(

    Ola



  • Hi @ogj, yes! And I know @jpsecher has done it our weather app.

    We have used a third-party library called: SmallJSONParser that does not do any memory allocation - which is important.

    There are a lot of third-party JSON parsers out there for C and C++. But we need one that is extremely memory efficient. That means it needs to be streaming. That is, it must handle situations where all JSON content cannot exist in memory. Mono's memory is limited, and JSON files can be quite large.

    SmallJSONParser can parse chunks of JSON data while you receive it from the network or a file. You provide it with JSON and ask it to return the next available token.

    A token is a JSON property like "name": "value". It will have the tokens name and value.

    Take a look at the Github page for the SmallJSONParser and see the examples. I can provide an example here later.

    I hope this is a help for now.



  • Thanks for the feedback.

    Ola



  • Hi @ogj

    Here is an example of using the minimal API of the SmallJsonParser:

    #include <stdio.h>
    #include <string.h>
    
    #include "SmallJSONParser.h"
    
    int main(int argc, const char **argv)
    {
      const char *jsonString = "{ \"hey\": \"you\", \"number\": 4 }";
    
      JSONParser parser;
      InitialiseJSONParser(&parser);
      ProvideJSONInput(&parser, (const void*) jsonString, strlen(jsonString));
    
      char buffer[100];
      memset(buffer, 0, 100);
      JSONToken token =  NextJSONToken(&parser);
      while (JSONTokenType(token) < OutOfDataJSONToken)
      {
          printf("Found token: %i\r\n", JSONTokenType(token));
          snprintf(buffer, token.end - token.start + 1, "%s", token.start);
          printf("Token value: %s\r\n", buffer);
    
          token = NextJSONToken(&parser);
      }
    
      printf("Parsing ended\r\n");
      
      return 0;
    }
    

    The program will print:

    Found token: 3
    Token value: {
    Found token: 1
    Token value: hey
    Found token: 1
    Token value: you
    Found token: 1
    Token value: number
    Found token: 2
    Token value: 4
    Found token: 4
    Token value: }
    Parsing ended
    


  • @stoffera
    Hi, Thanks.
    Can you please convert this to a sample code for Mono ?

    Ola



  • @ogj yes, that is easy.

    In Mono you still have access to libc and stdio, so we just substitute main with, say monoWakeFromReset:

    #include <stdio.h>
    #include <string.h>
    #include "SmallJSONParser.h"
    
    void AppController::monoWakeFromReset()
    {
      const char *jsonString = "{ \"hey\": \"you\", \"number\": 4 }";
    
      JSONParser parser;
      InitialiseJSONParser(&parser);
      ProvideJSONInput(&parser, (const void*) jsonString, strlen(jsonString));
    
      char buffer[100];
      memset(buffer, 0, 100);
      JSONToken token =  NextJSONToken(&parser);
      while (JSONTokenType(token) < OutOfDataJSONToken)
      {
          printf("Found token: %i\r\n", JSONTokenType(token));
          snprintf(buffer, token.end - token.start + 1, "%s", token.start);
          printf("Token value: %s\r\n", buffer);
    
          token = NextJSONToken(&parser);
      }
    
      printf("Parsing ended\r\n");
    }
    

    Just remember to copy the two files SmallJSONParser.h and SmallJSONParser.c to your mono project folder. You do not need to alter the Makefile, it will automatically compile the new files.

    NB: Since the printf output is sent over the serial port, you might miss some of the output, because monoWakeFromReset is run before the serial has a change to connect. Consider copying the parsing code to a separate function, that you trigger using a button or a timer.



  • @stoffera
    Hi, I also tried this yesterday, but I get an error:

    SmallJSONParser.h: In function 'void InitialiseJSONProvider(JSONProvider*, bool
    ()(JSONParser, void*), void*, void*, size_t)':
    SmallJSONParser.h:190:14: error: invalid conversion from 'void*' to 'uint8_t* {a
    ka unsigned char*}' [-fpermissive]
    self->buffer=buffer;

    Ola



  • Hi @ogj , that is weird!

    It works when I compile it on my computer (with clang). Well, on Mono our Arm GCC compiler seems a little more strict than clang.

    The issue is a bug in SmallJsonParser, that you can easily resolve. Goto line 188:

    static inline void InitialiseJSONProvider(JSONProvider *self,
    JSONInputProviderCallbackFunction *callback,void *context,uint8_t *buffer,size_t buffersize)
    {
    	self->callback=callback;
    	self->context=context;
    	self->buffer=buffer;
    	self->buffersize=buffersize;
    }
    

    Change the type of the buffer argument from void* to uint8_t* can the issue is fixed.

    Also, remember to encapsulate the #include statement in the C function exporter:

    extern "C" {
    #include "SmallJSONParser.h"
    }
    

    Hope this helps.



  • @stoffera
    Nice, now it works. Have a Nice weekend.

    Ola


  • administrators

    If you want a complete example of how to parse larger pieces of JSON data, try looking at the source code for the Weather app.

    A short explanation: The json::Json class lets you pick out values from a JSON document by using paths like /location/city. The actual data behind the JSON document is delivered by the SdCardByteBuffer class, which allows you to stream data to a file on the SD card, and later read it back in again. The Json class does not really know that it is reading from the SD card; instead it accepts any kind of buffer as long as the buffer implements the IByteBuffer interface, which abstracts away the nitty-gritty detail of buffering data. Likewise, the Wifi class relies the IByteBuffer interface to store the data received from the network.


Log in to reply