13 Abstractions

In this chapter we tackle one of the heavier subjects related to application design ie object orientation. Another way to think about this is abstraction in terms of basic operations like storage, moving, conversion to different formats as well as behaviors. Ada supports such abstractions rather elegantly. In particular combined with generic’s which enable a related form, gives developers a strong platform.

In two projectlets, we will tackle the many facets of abstraction.

13.0.1 Learning Objectives

Operational abstraction
Data Type abstraction
Combination of generics and abstraction
Data analysis and manipulation support tooling
Introductory networking

13.1 Projectlet: Logging

The first projectlet is about data logging from applications. Such logs are generated at sub second rates in realtime applications while it may be at a lower rate in say a DevOps support utility. The key goal of such logging is of course to post process and analyze. More often than not, in a system logging from different sub modules need to be collated for system level analysis and diagnostics. In more extreme cases, the modules may be distributed across a network. Different aspects of such logging are abstracted leaving enough scope for further expansion.

13.1.1 Logging API

Essentially the logging needs of applications can be outlined in the form of an API as in:

~/bin/codemd ../../toolkit/adalib/src/logging.ads -x API -l

0004 |    subtype message_level_type is Natural;
0005 | 
0006 |    CRITICAL      : constant message_level_type := 10;
0007 |    ERROR         : constant message_level_type := 20;
0008 |    WARNING       : constant message_level_type := 30;
0009 |    INFORMATIONAL : constant message_level_type := 40;
0010 |    function Image (level : message_level_type) return String;
0011 | 
0012 |    subtype Source_Name_Type is String (1 .. 6);
0013 |    Default_Source_Name : Source_Name_Type := (others => '.');
0014 | 
0015 |    subtype Message_Class_Type is String (1 .. 6);
0016 |    Default_Message_Class : Message_Class_Type := (others => '.');
0017 | 
0018 |    function Time_Stamp return String;
0019 | 
0020 |    procedure SendMessage
0021 |      (message : String; level : message_level_type := INFORMATIONAL;
0022 |       source  : String := Default_Source_Name;
0023 |       class   : String := Default_Message_Class);

The application can provide a message in text in addition to some attributes like severity, the source or function for each message. Thus messages from different threads can be interspersed yet distinguished from each other.

The messages have to be printed or somehow captured. The exact method of printing or collating is left unspecified or abstracted giving us the flexibility of different implementations.

~/bin/codemd ../../toolkit/adalib/src/logging.ads -x Dest -l

0032 |    type Destination_Type is abstract tagged record
0033 |       null;
0034 |    end record;
0035 |    procedure SetDestination (destination : access Destination_Type'Class);
0036 |    procedure SendMessage
0037 |      (dest   : in out Destination_Type; message : String;
0038 |       level  :        message_level_type := INFORMATIONAL;
0039 |       source :        String             := Default_Source_Name;
0040 |       class  :        String             := Default_Message_Class) is abstract;
0041 |    procedure Close (desg : Destination_Type) is abstract;
0042 | 
0043 |    type StdOutDestination_Type is new Destination_Type with record
0044 |       null;
0045 |    end record;

In the simplest application, the logs just get sent to the Standard_Output channel. An application may then generate a trace as follows:

~/bin/codemd ../../toolkit/examples/logs/src/logs.adb -x Simple -l

0028 |    procedure T2 (argc : integer) is
0029 |       myname : constant String := GNAT.Source_Info.Enclosing_Entity;
0030 |    begin
0031 |       if argc = 0
0032 |       then 
0033 |          logging.SetDestination (cstdout.handle'Access);
0034 |       end if ;
0035 |       Logging.SendMessage ("Message 1");
0036 |       Logging.SendMessage ("Critical ", logging.CRITICAL);
0037 |       Logging.SendMessage ("Error", logging.ERROR);
0038 |       logging.SendMessage ("Warning", logging.WARNING);
0039 |    end T2;

The above use case is the simplest with the application not needing any initialization with the logs going to Standard Output, resulting in:

~/bin/logs a

2025-02-06 10:36:54 ...... ...... [I] Message 1
2025-02-06 10:36:54 ...... ...... [C] Critical 
2025-02-06 10:36:54 ...... ...... [E] Error
2025-02-06 10:36:54 ...... ...... [W] Warning

With a minor additional initialization, with no change to the application, the logs get sent to a different destination in this case a UDP socket (port no 1056 on the same host localhost) destination. Presumably a log server captures the logs via an UDP socket and thus able to collate logs from many sources:

~/bin/codemd ../../toolkit/examples/logs/src/socklog.adb -x Socklog -l

0008 |    procedure T1 is
0009 |       myname  : constant String := GNAT.Source_Info.Enclosing_Entity;
0010 |       logsock : aliased logging.socket.SocketDestinationPtr_Type;
0011 |    begin
0012 |       logsock := logging.socket.Create (1_056, "localhost");
0013 |       logging.SetDestination (logsock);
0014 |       for i in 1 .. 10 loop
0015 |          Logging.SendMessage ("Message 1");
0016 |          Logging.SendMessage ("Critical ", logging.CRITICAL);
0017 |          Logging.SendMessage ("Error", logging.ERROR);
0018 |          logging.SendMessage ("Warning", logging.WARNING);
0019 |          delay 2.0;
0020 |       end loop;
0021 |    end T1;

13.1.2 Implementation

Internally of course different destinations require different handling. A socket destination requires a different initialization vs a text file destination. However, fundamental logic for creating a log is the same. This is achieved by designating an abstract, tagged data type ie Destination_Type and at runtime the appropriate implementation for the specific destination is invoked by dispatching.

~/bin/codemd ../../toolkit/adalib/src/logging.adb -x Dispatch -l

0053 |    procedure SendMessage
0054 |      (message : String; level : message_level_type := INFORMATIONAL;
0055 |       source  : String := Default_Source_Name;
0056 |       class   : String := Default_Message_Class)
0057 |    is
0058 |    begin
0059 |       SendMessage (Current_Destination.all, message, level, source, class);
0060 |    end SendMessage;

and for the socket destination in particular:

~/bin/codemd ../../toolkit/adalib/src/logging-socket.adb -x SockDest -l

0039 |    overriding procedure SendMessage
0040 |      (dest   : in out SocketDestination_Type; message : String;
0041 |       level  :        message_level_type := INFORMATIONAL;
0042 |       source :        String             := Default_Source_Name;
0043 |       class  :        String             := Default_Message_Class)
0044 |    is
0045 |       m       : aliased Message_Type;
0046 |       payload : Ada.Streams.Stream_Element_Array (1 .. m'Size / 8);
0047 |       for payload'Address use m'Address;
0048 |       last : Ada.Streams.Stream_Element_Offset;
0049 |    begin
0050 |       m.t   := Ada.Calendar.Clock;
0051 |       m.Seq := dest.Count + 1;
0052 |       m.l   := level;
0053 |       m.s   := source;
0054 |       m.c   := class;
0055 |       if message'Length > MAX_MESSAGE_SIZE then
0056 |          m.ml := MAX_MESSAGE_SIZE;
0057 |          m.mt := message (1 .. MAX_MESSAGE_SIZE);
0058 |       else
0059 |          m.ml                       := message'Length;
0060 |          m.mt (1 .. message'Length) := message;
0061 |       end if;
0062 |       GS.Send_Socket (dest.s, payload, last, To => dest.dest);
0063 |       if last /= m'Size / 8 then
0064 |          raise Program_Error with "Truncated datagram";
0065 |       end if;
0066 |       dest.Count := @ + 1;
0067 |    end SendMessage;

13.2 Projectlet: Tables

A totally different challenge occurs for example in the Data Analysis domain - analyzing huge amounts of related data most often in the form of tables. Most beautifully illustrated in the tidyverse and dplyr packages in the R framework. (Ref: https://tidyr.tidyverse.org. Similar support are available for Julia as well; thus the tools of choice in many big data applications particularly in Life Sciences/Healthcare. The challenge in this case is the nature of data items - sometimes real, sometimes timestamps, other times categorical. For a strongly typed language this is not very straightforward. The second projectlet of this chapter attempts to sketch a path to implementing such support in Ada.

~/bin/codemd ../../toolkit/adalib/src/tables.ads -x Abstract -l

0010 |    type ColumnType is abstract tagged record
0011 |       name : Unbounded_String;
0012 |    end record;
0013 | 
0014 |    procedure Append (col : in out ColumnType; value : String) is abstract;
0015 |    procedure Set
0016 |      (col : in out ColumnType; idx : Natural; value : String) is abstract;
0017 |    function Image
0018 |      (col : in out ColumnType; idx : Natural) return String is abstract;
0019 |    function Length (col : ColumnType) return Natural is abstract;
0020 |    procedure Remove (col : in out ColumnType; idx : Natural) is abstract;
0021 | 
0022 |    type ColPtrType is access all ColumnType'Class;
0023 |    --function Create (name : String) return ColPtrType is abstract;
0024 |    package TablePkg is new Ada.Containers.Vectors (Natural, ColPtrType);
0025 |    subtype TableType is TablePkg.Vector;

A table TableType is defined in the above to be a Vector for Columns. A ColumnType is an abstract data type with name a String being a common attribute for all its derivatives. Being a strongly typed language, any of the operations on elements of this column need to be data type specific. The generic support to the rescue:

~/bin/codemd ../../toolkit/adalib/src/tables.ads -x BasicTypes -l

0029 |    generic
0030 |       type T is private;
0031 |       with function Vfun (s : String) return T;
0032 |       with function Ifun (v : T) return String;
0033 |    package ColumnPkg is
0034 | 
0035 |       package ColumnValues_Pkg is new Ada.Containers.Vectors (Natural, T);
0036 |       subtype ColumnValuesType is ColumnValues_Pkg.Vector;
0037 | 
0038 |       type TColumnType is new ColumnType with record
0039 |          values : ColumnValuesType;
0040 |       end record;
0041 | 
0042 |       function Create (name : String) return ColPtrType;
0043 |       function Length (col : TColumnType) return Natural;
0044 |       procedure Remove (col : in out TColumnType; idx : Natural);
0045 |       function Get (col : TColumnType; idx : Natural) return T;
0046 | 
0047 |       procedure Append (col : in out TColumnType; value : String);
0048 |       procedure Set (col : in out TColumnType; idx : Natural; value : String);
0049 |       function Image (col : in out TColumnType; idx : Natural) return String;
0050 | 
0051 |    end ColumnPkg;

Unfortunately this solution does not work for all data types. For example, strings being of a varying size, need special handling. Similarly other data types such as Date and Time will require special handling. Overall architecture supports such specialized implementations:

~/bin/codemd ../../toolkit/adalib/src/tables.ads -x StrCol -l

0055 |    package StringColumnValues_Pkg is new Ada.Containers.Vectors
0056 |      (Natural, Unbounded_String);
0057 |    type StringColumnType is new ColumnType with record
0058 |       values : StringColumnValues_Pkg.Vector;
0059 |    end record;

where strings are converted to Unbounded_Strings to be stored in such a table. The implementation for the abstract routines Append, Image and so on perform the appropriate and necessary conversions:

~/bin/codemd ../../toolkit/adalib/src/tables.adb -x StrCol -l

0076 |    function Get (col : StringColumnType; idx : Natural) return String is
0077 |    begin
0078 |       return To_String (col.values.Element (idx));
0079 |    end Get;
0080 | 
0081 |    procedure Set (col : in out StringColumnType; idx : Natural; value : String)
0082 |    is
0083 |    begin
0084 |       StringColumnValues_Pkg.Replace_Element
0085 |         (col.values, idx, To_Unbounded_String (value));
0086 |    end Set;
0087 | 
0088 |    procedure Append (col : in out StringColumnType; value : String) is
0089 |    begin
0090 |       StringColumnValues_Pkg.Append (col.values, To_Unbounded_String (value));
0091 |    end Append;
0092 | 
0093 |    function Image (col : in out StringColumnType; idx : Natural) return String
0094 |    is
0095 |    begin
0096 |       return Get (col, idx);
0097 |    end Image;

13.2.1 Table Handling

Putting all of that together, the application needs to define the table:

~/bin/codemd ../../toolkit/examples/ptable/src/ptable.adb -x Table -l

0030 |    procedure T1 is
0031 |       tname         : constant String   := GNAT.Source_Info.Enclosing_Entity;
0032 | 
0033 |       atomic_number : tables.ColPtrType := integer_column.Create("AtomicNumber");
0034 |       symbol : tables.ColPtrType := tables.CreateStringColumn ("Symbol");
0035 |       name          : tables.ColPtrType := tables.CreateStringColumn ("Name");
0036 |       atomic_mass   : tables.ColPtrType := float_column.Create ("AtomicMass");
0037 |       pt            : tables.TableType;
0038 |    begin
0039 |       pt.Append (atomic_number);
0040 |       pt.Append (name);
0041 |       pt.Append (symbol);
0042 |       pt.Append (atomic_mass);
0043 | 
0044 |       tables.Load( Argument(1) , pt , "," );
0045 |       tables.Print(pt);
0046 |       tables.Save(Argument(1) & ".csv" , pt );
0047 |    end T1;
0048 |    -- CODEMD: END
0049 | begin
0050 |    if Argument_Count > 0 then
0051 |       T1;
0052 |    end if;
0053 | end Ptable;

and other operations to map the table to a CSV file are enabled:

~/bin/codemd ../../toolkit/adalib/src/tables-load.adb -x Load -l

0005 | procedure Load
0006 |   (filename : String; table : in out TableType; sep : String := ";")
0007 | is
0008 |    use type Ada.Containers.Count_Type;
0009 |    tblfile : Csv.File_Type;
0010 | begin
0011 |    tblfile := Csv.Open (filename, Separator => sep, FieldNames => False);
0012 |    loop
0013 |       for fld in 0 .. TablePkg.Length (table) - 1 loop
0014 |          declare
0015 |             --colname : String :=
0016 |             --  To_String (TablePkg.Element (table, Integer (fld)).name);
0017 |             colval  : constant String := Csv.Field (tblfile, Integer (fld) + 1);
0018 |          begin
0019 |             TablePkg.Element (table, Integer (fld)).Append (colval);
0020 |          end;
0021 |       end loop;
0022 |       if Csv.End_Of_File (tblfile) then
0023 |          exit;
0024 |       end if;
0025 |       Csv.Get_Line (tblfile);
0026 |    end loop;
0027 |    Csv.Close (tblfile);
0028 | end Load;

13.3 Stretch

In order to handle the potential loss of log records, the network channel could be modified to use stream sockets and applications can take advantage with no change in the application.
The file destination should be modified to create a new logfile every hour - totally transparent to the application.
Datalogs with binary data instead of textual representation will be a really powerful tool to build time series data from realtime applications - particularly at high data rates.
Additional data types for Date and Time will be needed.
Support for all the dplyr features should be added to make this feature complete in this package.

13.4 Sample Implementation

Repository: toolkit
Directories: example/logs example/ptable