16  The wide world

The ecosystem of Ada development centered around alire has vastly expanded the free resources available for learning or for practical use in our own applications. Some of the crates like https://alire.ada.dev/crates/gsl provide a conduit to full featured, open source libraries like https://www.gnu.org/software/gsl/ that are not in ada by building bindings. While it is definitely feasible to build such a library entirely from scratch in Ada, the bindings provide continued access to the brilliant minds that contribute to such libraries.

The subject of this projeclet is https://mosquitto.org; in particular applications utilizing the MQTT protocol. Starting from a thin binding to the mosquitto library which is automatically generated, a layer of Ada like interfaces (or packages) are developed. A simple application is developed to illustrate the features.

16.1 Learning Objectives

  • Steps involved in building bindings

  • Thin vs Thick or Thickish bindings

  • Communication patterns beyond client/server

  • Leveraging testing resources

16.2 Numerical Expression Captcha

The goal of this projectlet is to support https://en.wikipedia.org/wiki/CAPTCHA except the challenge will be numerical expressions expecting the responder to compute the answer prove that they are not an automaton. Since the numerical expressions are dynamically and randomly generated the answers are not static and thus provides a reasonable challenge.

16.2.1 High Level design

The support is separated into a quizzer which is responsible for challenging the user and providing the necessary permissions. The quizzer however does not generate the numerical expression - relying on a poser of quizzes to provide a problem as well as the solution. Such a communication utilizes the MQTT protocol, in particular the mosquitto library opening up many possibilities.

In this projectlet, the communication patterns utilized is very simple - periodic broadcast. Every minute or so the poser broadcasts an automatically generated puzzle and its solution. The quizzer listens for such a broadcast for challenging the user till a successful solution is provided by the user.

16.2.2 Binding generation

The first step in this effort is to generate a thin binding to the library - which translates the basic definitions from presumably the native C specifications to Ada.

g++ -fdump-ada-spec /opt/homebrew/include/mosquitto.h

On a MacOS, for example the following generates the basics :

arm_types_h.ads arm_utypes_h.ads mosquitto_h.ads stddef_h.ads stdint_h.ads sys_upthread_upthread_types_h.ads sys_utypes_h.ads sys_utypes_uint16_t_h.ads sys_utypes_uint32_t_h.ads sys_utypes_uint64_t_h.ads sys_utypes_uint8_t_h.ads sys_utypes_uintptr_t_h.ads sys_utypes_uu_int16_t_h.ads sys_utypes_uu_int32_t_h.ads sys_utypes_uu_int64_t_h.ads sys_utypes_uu_int8_t_h.ads sys_utypes_uuintptr_t_h.ads utypes_uintmax_t_h.ads utypes_uuint16_t_h.ads utypes_uuint32_t_h.ads utypes_uuint64_t_h.ads utypes_uuint8_t_h.ads utypes_uuintmax_t_h.ads

Of these package specifications, only *mosquitto_h.ads is specific to the library of interest while others all are the specifications of the package.

head -25 ../../bindings/adamosquitto/work/mosquitto_h.ads
pragma Ada_2012;

pragma Style_Checks (Off);
pragma Warnings (Off, "-gnatwu");

with Interfaces.C; use Interfaces.C;
with Interfaces.C.Strings;
with System;
with Interfaces.C.Extensions;
with stddef_h;
with utypes_uuint8_t_h;
with utypes_uuint16_t_h;
with utypes_uuint32_t_h;

package mosquitto_h is

   LIBMOSQUITTO_MAJOR : constant := 2;  --  /opt/homebrew/include/mosquitto.h:67
   LIBMOSQUITTO_MINOR : constant := 0;  --  /opt/homebrew/include/mosquitto.h:68
   LIBMOSQUITTO_REVISION : constant := 20;  --  /opt/homebrew/include/mosquitto.h:69
   --  unsupported macro: LIBMOSQUITTO_VERSION_NUMBER (LIBMOSQUITTO_MAJOR*1000000+LIBMOSQUITTO_MINOR*1000+LIBMOSQUITTO_REVISION)

   MOSQ_LOG_NONE : constant := 0;  --  /opt/homebrew/include/mosquitto.h:74
   MOSQ_LOG_INFO : constant := (2**0);  --  /opt/homebrew/include/mosquitto.h:75
   MOSQ_LOG_NOTICE : constant := (2**1);  --  /opt/homebrew/include/mosquitto.h:76
   MOSQ_LOG_WARNING : constant := (2**2);  --  /opt/homebrew/include/mosquitto.h:77

16.2.3 Prettier Package

Beginning with renaming the package from mosquitto_h to mosquitto, some simple heuristics provide a prettier interface that renders the users more readable:

head -25 ../../bindings/adamosquitto/src/mosquitto.ads
pragma Ada_2012;

pragma Style_Checks (Off);
pragma Warnings (Off, "-gnatwu");

with Interfaces;   use Interfaces;
with Interfaces.C; use Interfaces.C;
with Interfaces.C.Strings;
with System;
with Interfaces.C.Extensions;

package mosquitto is

   LIBMOSQUITTO_MAJOR    : constant :=
     2;  --  /opt/homebrew/include/mosquitto.h:67
   LIBMOSQUITTO_MINOR    : constant :=
     0;  --  /opt/homebrew/include/mosquitto.h:68
   LIBMOSQUITTO_REVISION : constant :=
     20;  --  /opt/homebrew/include/mosquitto.h:69
   --  unsupported macro: LIBMOSQUITTO_VERSION_NUMBER (LIBMOSQUITTO_MAJOR*1000000+LIBMOSQUITTO_MINOR*1000+LIBMOSQUITTO_REVISION)

   MOSQ_LOG_NONE : constant := 0;  --  /opt/homebrew/include/mosquitto.h:74
   MOSQ_LOG_INFO        : constant :=
     (2**0);  --  /opt/homebrew/include/mosquitto.h:75
   MOSQ_LOG_NOTICE      : constant :=

As shown below, the macros for error codes have been shortened to remove redundant and repetitive phrases:

~/bin/codemd ../../bindings/adamosquitto/src/mosquitto.ads -x Errors -l
0053 |    subtype mosq_err_t is int;
0054 |    AUTH_CONTINUE         : constant mosq_err_t := -4;
0055 |    NO_SUBSCRIBERS        : constant mosq_err_t := -3;
0056 |    SUB_EXISTS            : constant mosq_err_t := -2;
0057 |    CONN_PENDING          : constant mosq_err_t := -1;
0058 |    SUCCESS               : constant mosq_err_t := 0;
0059 |    NOMEM                 : constant mosq_err_t := 1;
0060 |    PROTOCOL              : constant mosq_err_t := 2;
0061 |    INVAL                 : constant mosq_err_t := 3;
0062 |    NO_CONN               : constant mosq_err_t := 4;
0063 |    CONN_REFUSED          : constant mosq_err_t := 5;
0064 |    NOT_FOUND             : constant mosq_err_t := 6;
0065 |    CONN_LOST             : constant mosq_err_t := 7;
0066 |    TLS                   : constant mosq_err_t := 8;
0067 |    PAYLOAD_SIZE          : constant mosq_err_t := 9;

The functions (and procedures) have been renamed in the specification though the Import pragmas provide the complete C external name.

~/bin/codemd ../../bindings/adamosquitto/src/mosquitto.ads -x Functions -l
0145 |    function alloc
0146 |      (id  : Interfaces.C.Strings.chars_ptr; clean_session : Extensions.bool;
0147 |       obj : System.Address)
0148 |       return Handle  -- /opt/homebrew/include/mosquitto.h:314
0149 |    with
0150 |      Import => True, Convention => C, External_Name => "mosquitto_new";
0151 | 
0152 |    procedure destroy
0153 |      (mosq : Handle)  -- /opt/homebrew/include/mosquitto.h:327
0154 |    with
0155 |      Import => True, Convention => C, External_Name => "mosquitto_destroy";

16.2.4 Thickish Layer

Though the above package is functional and could support most applications, it is not Ada friendly. For example all strings are expected to be null terminated as is the norm in the C world. We help ourselves by building a family of packages which provide a lot more Adaish interfaces.

~/bin/codemd ../../bindings/adamosquitto/src/mqtt.ads -x Adaish -l
0005 | package mqtt is
0006 | 
0007 |    verbose : Boolean := True;
0008 | 
0009 |    INITIALIZATION_FAILURE  : exception;
0010 |    HANDLE_CREATION_FAILURE : exception;
0011 |    CONNECT_FAILURE         : exception;
0012 | 
0013 |    test_server           : constant String  := "test.mosquitto.org";
0014 |    -- These use cleartext messages
0015 |    test_server_port : constant Integer := 1_883;       -- No Username, password
0016 |    test_server_port_auth : constant Integer := 1_884;  -- Authenticated. username+password
0017 | 
0018 |    function ErrorMessage (code : mosquitto.mosq_err_t) return String;
0019 | 
0020 |    function Create (id : String := "") return mosquitto.Handle;
0021 |    procedure Destroy (h : mosquitto.Handle ) renames mosquitto.destroy;

The burden of handling strings is eliminated with this layer. Similarly status codes are converted into Exceptions.

Further, the interfaces required for a producer being distinct from those of the consumer of the information, individual packages provide a friendlier interface:

16.2.4.1 Producer

~/bin/codemd ../../bindings/adamosquitto/src/mqtt-publisher.ads -x Publisher -l
0002 | package mqtt.Publisher is
0003 | 
0004 |    procedure Connect
0005 |      (h        : mosquitto.Handle; server : String := test_server;
0006 |       port : Integer := test_server_port; will_topic : access String := null;
0007 |       will_msg : access String := null; keepalive : Integer := 60);
0008 | 
0009 |    procedure Start (h : mosquitto.Handle);
0010 | 
0011 |    procedure Publish
0012 |      (h   : mosquitto.Handle; topic : String; payload : String;
0013 |       qos : Integer := 2; retain : Boolean := False);
0014 | 
0015 | end mqtt.Publisher;

16.2.4.2 Consumer

~/bin/codemd ../../bindings/adamosquitto/src/mqtt-subscriber.ads -x Subscriber -l
0002 | package mqtt.subscriber is
0003 |    SUBSCRIBE_ERROR : exception;
0004 |    type MessageHandler is
0005 |      access procedure (msg : access constant mosquitto.mosquitto_message);
0006 | 
0007 |    procedure Connect
0008 |      (h     : mosquitto.Handle; msghandler : not null MessageHandler;
0009 |       topic : String; server : String := test_server;
0010 |       port  : Integer := test_server_port; keepalive : Integer := 60);
0011 |    procedure Start (h : mosquitto.Handle);
0012 | end mqtt.subscriber;

16.3 Projectlet Quiz

With the layers of support outlined above, a producer of the quiz or publisher in MQTT terminology becomes simple:

~/bin/codemd ../../bindings/adamosquitto/examples/pquiz/src/pquiz.adb -x Publisher -l
0046 |    h := mqtt.Create ;
0047 |    mqtt.Publisher.Connect(h) ;
0048 |    mqtt.Publisher.Start(h);
0049 | 
0050 |    for i in 1..10
0051 |    loop
0052 |       delay 60.0;
0053 |       pexpr := numexpr.Generate(2);
0054 |       pubtime := Ada.Calendar.Clock ;
0055 |       declare
0056 |          payload : String := Ada.Calendar.Formatting.Local_Image(pubtime) & "::" &
0057 |                                          numexpr.Image(pexpr) &
0058 |                                          " == " &
0059 |                                          numexpr.Image( numexpr.Evaluate(pexpr) );
0060 |       begin
0061 |          mqtt.Publisher.Publish(h,"pquiz" , payload);
0062 |          Put_Line(payload);
0063 |       end ;
0064 |    end loop ;

and the subscriber with a little help from an external package to handle each message received:

~/bin/codemd ../../bindings/adamosquitto/examples/quizzer/src/quizzer.adb -x Quizzer -l
0008 | procedure Quizzer is
0009 |    h : mosquitto.handle;
0010 | begin
0011 |    h := mqtt.Create;
0012 | 
0013 |    mqtt.subscriber.Connect(h,
0014 |                            handlers.strPayload'Access,
0015 |                            "pquiz");
0016 |    mqtt.subscriber.Start(h);
0017 | 
0018 |    for i in 1..10
0019 |    loop
0020 |       Put_Line("Waiting");
0021 |       delay 60.0 ;
0022 |    end loop ;
0023 | end Quizzer;

16.4 Stretch

  • Instead of a periodic broadcast, convert the interaction to a query/response interaction which is more conventional

  • Introduce additional topics in MQTT parlance - providing different types of quizzes - word puzzles for instance

  • The test servers used in the examples are meant for testing purposes. Setup the brokers dedicated to this application.

16.5 Sample Implementation

Repository: adamosquitto
Directory: examples/pquiz
Directory: examples/quizzer
Directory: docker