Cloudgizer

Cloudgizer is a tool for building web applications as Apache modules, with emphasis on performance, small-footprint, and more productive and safer programming in C.

Current stable version is 1.2 (as tagged in git repository).

This site has been last updated on Fri Jul 20 15:32:58 MST 2018.

This is the documentation for the latest Cloudgizer code. If you download older tagged versions from the git repository, each comes with 'docs' directory that contains the documentation matching that version.



About Cloudgizer
    Example source code - learn by example
    License
    Source code, reporting issues
    Hello world
Basics
    The big picture
    Basic components
    The architecture
    Structure of an application
    Hello World
Download and Installation
    Install Cloudgizer
    Get started with the Example application
    Your own application
        Packaging
        Deploying
        Developing and building application
    Smoke tests
Getting started
    Configuration, database and debug files
        Configuration file
        Database file
        Debug file
    How to write code
        Markup code block
        Markup usage
        Line continuation and whitespaces
        A typical program
        Errors in your program
    CLD utility
Processing of requests
Constructs
    Comments
    Outputting data
        Whitespaces and empty lines
        Semicolon
        Encoding output
        Ignoring markup
        Outputting preprocessor data to a file
        Outputting HTTP header for a typical web page
        Defining, setting and outputting integers
        Outputting data from a command line program
    Embedded C code
    URL input parameters (GET and POST)
    Cookies, setting and getting
    Database queries
        Basic usage of queries
        Database transactions
        Column Length
        When query has no results
        Force query to produce a single empty row
        Current row in a query result set
        Number of rows and columns in a query result set
        Number of affected rows in DML
        Input parameters to queries
        Column names
        Nesting queries
        Using multiple query texts at run-time (conditional queries)
        Dynamic queries
        Dynamic queries when table structure is unknown
        Result-set data array
        Using query results more than once
        Copying query text in multiple places safely (query shards)
        Re-using query text in multiple places (query fragments)
        Loop control in result set
        Error handling in executing queries
        Getting auto_increment value
        NULL and result column data types
    Executing Cloudgizer program from command line
    Executing external programs
    Loading URL content (calling a web page) programmatically
    Strings
        Defining and copying strings
        Appending string
        Search and replace string
        Constructing complex strings
    Program flow
        Loops
        Conditional statements (if, else...)
    Files
        File storage
        Permissions and ownership
        Uploading files
        Reading a whole file
        Writing a whole file or appending to a file
        Copying a file
    Web address of server
    Error handling in general
    Include files
    Debugging your code
        Tracing and debugging options
        Finding where program crashed
        Enabling debugging information
        Which shared libraries are loaded?
API
    Memory allocation
        Allocation and garbage collection
        Get size of memory allocated for variable
        Check if allocated memory is valid, overwritten or underwritten
    Configuration parameters
    Tracing and debugging
        Write to trace file
        Which trace file is being written to?
        Using custom tag value for debugging
        Linting your HTML code dynamically
    URL input
        URL input parameters
        Is URL referrer from the same site?
        URL string that is being processed
        URL parameters manipulation
    Cookies
        Data type for storing
        Cookies: get, set, delete, find
    Global data
    Encoding and Decoding, URL encoding, Web encoding, Base64 encoding
    Numbers
        Create string from integer
        Check if string is a positive integer or a number
    Execution of external programs, piping from one program to another
    Output data and HTTP header
        Outputting HTTP header, custom HTTP response headers
        Encoding data for output
        Output text without breaks
        Disable and Enable HTML output, status of enablement
        Send files to the client and output HTTP headers for files
        File not found (404)
        Flush output
    Create new database document and a new associated file
    Strings
        Initialize, Copy, Append strings
        Output data to string
        Change case of strings (upper, lower)
        Substrings
        Trim string
        Break down string into an array of strings based on a delimiter
    Clear Unused variable error
    Error reporting
    Encryption and hashing
        Hashing
        Encrypting data
        Decrypting data
        Generating random string
    Storing and retrieving data in a sequential list (FIFO)
    Executing Cloudgizer program from command line (Batch processing)
        Using batch processing
        Exit code for batch processing
    POST URL (web call)
    Files
        File storage
        Get file size
        Copy files
        Read entire file into a string variable
        Write whole file from a string
        Append to file from a string
        Lock a file
        Get application home directory
        Check if directory
        Temporary files
    Send email
    Queries and Transactions
        SELECT from the database
        Get value from auto-increment column
        Execute any SQL other than SELECT, including DDL statements
        Check for open transaction
    Get miscellaneous
        Get web address for forms and links
        Get Base URL of server
        Get version
        Get current, past or future time, including GMT
        Get current time
        Get application name
        Get environment variable (web or command shell)
Installing from source code
    Cloudgizer
    Example Application

About Cloudgizer

Cloudgizer is a tool for building web applications as Apache modules in C programming language.

Example source code - learn by example


Browse the source code for both Example application and Rentomy to learn by example, get a feel for developing with Cloudgizer and to understand better the usage patterns of markups and API.

License


Cloudgizer is licensed under Apache License, Version 2.0.

Source code, reporting issues


Source code is available at Cloudgizer Application git repository.

Older Cloudgizer versions stable enough for download and installation are tagged in git - in this case you'd be compiling from source - see here for details.

Report issues or questions - go to Issue tracker.

Hello world


Here is a program that outputs 'Hello World!' to the browser (and here' a database-enabled one):
#include "cld.h"

void home()
{
   /*<
    output-http-header

    Hello World!
   >*/
}

Basics

The big picture


Cloudgizer is a tool for building web applications as Apache modules in C language enhanced with simple markup, with emphasis on performance, small-footprint, and more productive and safer programming in C.

The programmer writes simple markup language mixed with C code, which is then translated entirely into C code and compiled natively as Apache module. The resulting application is fast and takes less memory, as there are no interpreters or virtual machines.

Features include easy markups to use MariaDB database, HTML input parameters, cookies, simpler outputting of web pages, files storage and manipulation, encryption, encoding, program execution, web calls, safer and easier string manipulation etc. - the list is too long to place in one sentence. Overall Cloudgizer does a lot of stuff for you that you'd otherwise need to do yourself.

A memory garbage collection system and memory overwrite/underwrite detection comes in handy for program stability. The same goes for string and memory handling markups to help write applications that won't crash.

Also included is an application packaging system and an automated application installer. This makes rollout of products and release cycle more manageable.

Cloudgizer source files have extension .v.

Cloudgizer pre-compiler (cld program) will turn your .v files into .c files, ready for compilation as pure C programs.

Then, your program will be compiled and linked with Apache web server on RH/Centos systems. It links with Apache as an Apache module in a "prefork" configuration. It does the work of communicating with Apache, and it makes it easier to write high-performance/small-footprint web programs in C.

Cloudgizer is not designed to be thread-safe as it works in a "prefork" configuration of Apache.

You can also build command-line programs. The same program can serve as both command-line utility and a web program linked with Apache.

Cloudgizer works with RedHat/Centos 7 operating system, Apache web server and mariaDB database.

Basic components


These are the files installed:


All scripts, the makefile and the command line utility are located in /usr/bin directory. Include file is in /usr/local/include/cld directory. Libraries are in /usr/local/lib/cld directory.

The architecture


Apache server is the container which holds your Cloudgizer applications as Apache modules. The module mechanism allows for fast response times and very little overhead.

Apache server can house many Cloudgizer applications, each being a separate Apache module. These separate modules are generally entirely different applications.

Apache handles the infrastructure part while your Cloudgizer code makes up the applications themselves.

All Cloudgizer applications are hosted under a single Linux user, with each application within its own subdirectory under the user's home directory. The Apache server user is set to this Linux user during the application installation - it is the currently logged-on user during installation. Make sure to install all applications under the same Linux user. Set the Apache's DocumentRoot to be accessible by this user.

Each Cloudgizer application has its own database schema within a MariaDB database server.

Structure of an application


Each application is installed in its own directory, known as the application home directory. If application name (given by CLD_APP_NAME in appinfo file during installation) is 'your_app_name', then application home directory is /home/user/your_app_name.

The following are important files your project must have - they all have default implementation in the Example Application:


In short, you must have the following implemented for an application to work:


The rest is you implementing the handler functions for different kinds of requests. To get started without writing everything from scratch, you should take the Example application and change it into your own.

Hello World


This is the Hello World example that is included in the Example application. It reads data from the database and displays it in the browser. It exemplifies the coding style of Cloudgizer, which is C code with markup. Markup code is enclosed in /*< and >*/ tags:
#include "cld.h"

void home()
{
   /*<

    output-http-header

    Hello World!

    run-query#get_db="select hi from hello"
       query-result#get_db, hi as define db_result
       print-web db_result
       <br/>
    end-query
    <hr/>

    >*/
}

Download and Installation

You need RedHat/Centos 7 on a dedicated server or a VPS (virtual private server) and Internet connectivity. Note that the binary distribution is compiled for x86_64 platform only.

Install Cloudgizer

# Install wget if you don't already have it:
sudo yum -y install wget

# Get installation file for the latest stable version:
wget "https://zigguro.org/cloudgizer/cloudgizer.tar.gz" -O cloudgizer.tar.gz

# Unpack installation file:
sudo rm -rf cloudgizer
tar xvfzm cloudgizer.tar.gz
cd cloudgizer

# Setup cloudgizer. You'll be asked questions (regarding database installation and Postfix install) - each stage of installation will be clearly marked:
sudo ./setup

# Verify installation, run cld tool - you should see help:
cld

# Check documentation, it's all in a single web page (like this one):
ls docs/index.html
Note: Setup process will create MariaDB database and you will be asked to create root database password. The setup will also install other needed software and also setup Postfix for sending emails using TLS encryption. If you are only upgrading (for example due to a bug fix in Cloudgizer), then you may run setup_cld only instead of setup script above.

To get started on writing your own application using Cloudgizer, first install the Example application and build on it. Once you create your own application, you would package it and then install it on customer's site. For these steps, see further in this documentation.

Get started with the Example application

Once you have installed Cloudgizer, get started by installing the Example application. This is required to build an application of your own.
#Choose a user for Cloudgizer applications (create one if needed). We will use 'cld' as user name. Login as that user. Apache user will be changed to this user during installation.
su - cld

# Get Example application
wget "https://zigguro.org/cloudgizer/example.tar.gz" -O example.tar.gz

# Unpack Example application
sudo rm -rf deploy
tar xvfzm example.tar.gz
cd deploy

#Open file appinfo. This file describes the application. Set these variables:

#After setting these variables, run:
cldgoapp create
To test the installation, in browser, go to these two URLs.
// you should see 'Hello World!' and 'Hi from the database!'.  
myserver/go.example?page=home

// Sample stock ticker example. The first link adds (or updates) the stock price, the second shows current ticker list:
myserver/go.example?page=stock&action=add&stock_name=RHT&stock_price=1000
myserver/go.example?page=stock&action=show

// this will run current Cloudgizer smoke tests.
myserver/go.example?page=test_cld&test_id=all  
In general, the URL's path is always go.your_app_name, where your_app_name is the application name, in this case 'example'. Application name is always the value of CLD_APP_NAME in appinfo file.

After you have successfully installed the Example application, use it as a base code for your own. You'd probably remove all request handlers except Hello World and start from there.

Your own application

Packaging

Once your application is ready to be installed at customer site, package the application. The appinfo file is the configuration file for the installation process, and it should contain any variables your installation needs. Do not write appinfo from scratch - always start from the file provided in the Example application and build on it. Script cldpackapp will package your application by producing your_app_name.tar.gz file your customer can install by using cldgoapp.

Most of the stuff done during installation is the same for any application. The application-specific part of installation is done through create.sh file. This is a script you provide in which you can setup anything that installation process can't do, for example you can create database tables or any other custom setup your application needs. If the application is being updated (as opposed to created for the very first time), you would create update.sh file instead and write your application-specific code there. You can use CLD_APP_INSTALL_DIR in these shell scripts to locate files in the installation.

appinfo file is a configuration file for the installation. It always has these variables: CLD_SERVER, CLD_APP_NAME, CLD_EMAIL, CLD_DB_ROOT_PWD and CLD_DB_APP_PWD - these variables are set by the customer during installation. Your responsibility is to set the following variables in appinfo:

Once you have appinfo file as described above, as well as create.sh/update.sh files, create a deployment file:
cldpackapp
This will create your_app_name.tar.gz installation file.

Check out the source code of the Example application.

Deploying

At customer's site unpack the installation file:
sudo rm -rf deploy
tar xzvfm your_app_name.tar.gz
cd deploy

# Edit appinfo file, set these variable to match customer's environment:

#
# Use one of the following to install:
#

# to create application
cldgoapp create

# to update application
cldgoapp update
Application is always deployed from binaries. If you distribute source code, then the end-user can recompile the source code by using cldbuild script.

Any variable that has PWD in the name will be cleared upon installation for security reasons.

Developing and building application


If you are a developer, you'd install the Example application first and then build your application by modifying it to suit your needs.

To build an already installed application, use cldbuild followed by restarting the web server:
cldbuild
sudo service httpd restart
Use this if you're developing Cloudgizer applications, or in testing.

You would use the above cldbuild followed by web server restart whenever you change your source code files, in order to test the changes.

Most of the time you'd change your source files only (such as .v files). From time to time you'd make more structural changes as your application grows. For example, when changing the database tables, adding or removing files (such as CSS, images, cron jobs) etc., make sure you test your installation. This means your create.sh (and update.sh for upgrades) must be reflecting all those changes in the deployed application. Use cldpackapp to package your application, copy it to a test environment and test your application from scratch using cldgoapp.

See more details on packaging and deploying.

Smoke tests

A battery of tests designed to shake out any obvious issues is incorporated in the Example application. Install it and run as prescribed - all tests run and display results in web browser. Typically each test will show OKAY if succeeded, or otherwise describe what is the desired outcome.

You can see what the smoke test output looks like live at: https://zigguro.org/go.example?page=test_cld&test_id=all. There are othere examples within the Example application as well.

Getting started

Configuration, database and debug files

These files direct Cloudgizer in some important aspects. Note that these are setup by cldgoapp script based on your appinfo file. cldgoapp script installs a Cloudgizer application. Your own create.sh script (a customizable part of installation) can further change these files.

Configuration file

A configuration file (named config) in the application home directory typically has something similar to this
   version=7
   web_address=http://192.168.0.11
   email_address="Hellocld" <root@localhost>
   application_name=Hellocld
   max_upload_size=10000000
   mariadb_socket=/var/lib/mysql/mysql.sock
   ignore_mismatch=no
version determines the application version. Typically it is used in constructed URL to force refreshment of cached files, but it can be used for any other versioning purpose.
web_address contains the server address where the application runs on, and is a base URL for Cloudgizer requests. You can use http:// or https://.
email_address is the email where status emails would be sent (for example in case of a program crash) and it can be used for any other emailing purpose by the application.
application_name is the name of application - it can be any name that is 16 bytes or smaller, composed of alphanumeric characters and  underscore, cannot start with a digit and cannot be 'deploy'.
max_upload_size is the maximum size of an upload file - uploading larger file will invoke predefined  file_too_large function, implemented by you.
mariadb_socket is the database identification, a means to connect to the database.
ignore_mismatch is by default "no", meaning that if shared library used to build application doesn't match what's installed on deployment server, stop the program. If "yes", skip this check and proceed. Use "yes" with caution and only if you know why you're doing it.

You can also define user parameters, which are always precedeed by _ (an underscore).

You can use all of these parameters in your code, see configuration parameters API.
If you need to use a Cloudgizer application home directory (for example in a user parameter), use tilde (~) sign, for example:
_my_user_directory = ~/user_dir

Database file

.db file is located in the application's home directory and it contains database login information. By default its permissions are set to 600 - do not change this. It has these four lines:
   localhost
   your_app_name (i.e. the CLD_APP_NAME from appinfo, the database user name)
   app_db_password ( i.e. the password you chose,  CLD_DB_APP_PWD)
   your_app_name (i.e. the name of the database created, which is the same as the database user name, which is CLD_APP_NAME in appinfo)

Debug file

A debug file (named debug in trace directory) determines the application behavior with respect to debugging. Take a look at debug file options for more details.

How to write code

Markup code block

Markup code block is written between /*< and >*/. It is a comment technically, but cld utility will parse it and generate C code for it. For example:
void my_func ()
{
   char *my_var = "hello";

   //
   // The following comment-block is Cloudgizer code.
   // It is processed by cld command line tool to create C code.
   // In this case very simple text outputting.
   //
   /*<
   Any free text that's not a markup is output.
   <hr/>
   print-web my_var
   <hr/>
   >*/
}
In this example, the output of the program (meaning output to web browser) is:

Any free text that's not a markup is output.
hello


The actual C code generated by Cloudgizer for this will look like:
void my_func ()
{
   char *my_var = "hello";

   cld_printf (CLD_NOENC, "Any free text that's not a markup is output.");
   cld_printf (CLD_NOENC, "<hr/>\n");
   cld_printf (CLD_NOENC, "%s", my_var);
   cld_printf (CLD_NOENC, "<hr/>\n");
}
So you can write the web output by simply writing the HTML code (or any other code, such as Javascript, or anything else that works for you). In this documentation, we'll talk about "HTML code" for simplicity, but when we say "HTML code" it means any kind of web code.

Anything else that is not HTML code, you will use markups such as print-web to output variables, run-query# to run database queries, input-param to get URL GET/POST input parameters etc.

If /*< and >*/ are not feasible, you can use /*CLD_BEGIN and CLD_END*/ as markup boundaries.

The markup beginning (i.e. /*<) must be the beginning of the line and markup ending (i.e. >*/) must be the ending of the line.

Markup usage


The markup (such as print-web) can be used in-between <? and ?> or without them in most cases, but not all. For example:
print-web my_var
is just fine on a single line. But if you want to combine it with other output on the same line, as in:
This is variable <?print-web my_var?>
then you must write the markup within <? and ?>. You can also always use these brackets if you prefer - if you are starting with Cloudgizer, it may help you catch errors sooner. Consider if you wrote:
printa-web my_var
In this case, printa-web is not a valid markup, even though you intended it to be. The output from this will be:

printa-web my_var

i.e. the literal output of what you wrote.  But if you wrote:
<?printa-web my_var?>
you would get an error. One might say that seeing your code in the output is a good enough way to find the error in fix it, but using markup will detect an error earlier - however the code might look a little less verbose.  

Line continuation and whitespaces


To continue a line of code over several lines, use a backslash (\). It can be used anywhere on the line, including in the middle of a string. Example:
run-query#my_query = "select name \
                           from employee \
                           where name like '%'"
is the same as:
run-query#my_query = "select name from employee where name like '%'"
Use only spaces as separators in markup code, i.e. do not use tabs and other whitespaces. A line may begin and end with tabs but do not use tabs within markup code itself.

A typical program


Example application provided with installation provides a good starting point. You can take it and grow it into your application, it is meant to be used that way.

Generally, you will get a request via cld_handle_request function. You implement this function (example is provided), and based on some input parameter (typically a page name specified in a URL), you'd call other functions. Most of these other functions would be implemented in their own separate files, thus typically each page would have its own implementation file.

These other functions are conceptually just like Hello World example. They usually query database and do some work for you. They may write files, read files, process data, generate HTML output, show images and documents and anything else you need.

You will notice that you generally shouldn't free the memory you allocate - it will be safely freed at the end of a request. Most of the time you won't need to allocate any memory at all, and that's a good thing. The markups chosen are what's usually needed. If something isn't there, you can use the API provided. If that's not enough, write your own C code for it - you can do just about anything in C.

If you allocate memory on your own always use Cloudgizer memory allocation API such as cld_malloc for example, since all memory allocated this way is automatically released at the end of the request.

Errors in your program


Cloudgizer tries to find as many errors in your program as possible. This also means uncovering some errors that might otherwise find their way into run-time. For example, all your non-dynamic SQL (which hopefully is all the SQL you'd ever use) is checked at run-time with the database, so for example syntax errors and using columns that do not exist are likely to be caught at compile time. Cloudgizer will try to figure out if you forgot to use c markup for the C code within the markup, which can cause code not to execute (and instead display) if you've forgotten it.

The code compiled by C compiler is generated from your original .v file. The error reporting from the C compiler (gcc) will show line numbers from your original code, not the generated code, sparing you the hunt for the actual line in .v file that caused an error.

At run-time, you'll have a number of facilities available for debugging, which is explained here in this documentation. You can learn how to use gdb to debug your program, how to check the validity of your generated XHTML at run-time, how to use tracing etc.

CLD utility


cld utility translates your .v file into .c file which is then compiled and linked with your program. If you type cld at the command line, you will get this help:

   Name
   
   Cloudgizer code generator, markup language and application server API, version [1.2]
   
   Description
   
   Cloudgizer is a tool for building Web applications in C that run as modules on Apache web server on RedHat/Centos. It supports mariaDB database by using LGPL mariaDB client that enables connectivity to mariaDB database. Each application runs under the same Apache web server user, under its own directory (i.e. application's home directory).
   
   Synopsis
   
   cld [<input-file-name.v>] [<command-line-options>]
   
   Options
   
   -help
       Display this help.
   
   -out <output-file-name.c>
       Write generated code to output file <output-file-name.c>. If this option is not used, generated code is written to stdout (standard output).
   
   -main
       Generate main C code. This option cannot be used when <input-file-name.c> is specified, i.e. either C code is generated for input-file-name.c or the main() function C code is generated.
   
   -cmd
       Generate C code for use as a standalone program (a command line program), rather than as an Apache module (Apache Mod) program which is the default. This option can only be used together with -main.
   
   -mariasock <socket-file-location>
       Specify the location of the mariaDB socket file, used by the database server (socket option in my.cnf).
   
   -v
       Print out verbose information about what is being done.
   
   -urlencode <string>
       Prints URL encoded <string>.
   
   -webencode <string>
       Prints web encoded <string>.
   

   COPYRIGHT AND LICENSE
   
   Copyright (c) 2017 Zigguro LLC (on the Web at https://zigguro.org/cloudgizer).
   Cloudgizer is free Open Source Software licensed under Apache License 2. Cloudgizer is "AS IS" without warranties or guarantees of any kind.

Processing of requests


The cld_handle_request function (the "request processor") is called from generated code to handle incoming requests and it must be implemented. The Example application already has an implementation.

A simple implementation may look like:
void cld_handle_request()
{
   /*<
   input-param page
   
   if-string page="home"
       c home ();
   else-if-string page="other"
       c other ();
   else
       report-error "Unrecognized page %s", page
   end-if
   >*/
}

The request processor function is called both for web requests (through Apache module mechanism), or for command line program execution. You can use any input parameters to differentiate input requests ('page' is used here as an example only).

Typically file cld_handle_request.v is created to hold the request processor code.

Constructs


The markups are always within /*<  ... >*/ - that's a markup block. They are parsed by cld utility and C code is produced where comment was before. You will see that you can also have pure C code within /*< ... >*/, so while /*< ... >*/ is within a C code already, you can have C code again within it. It's useful because it's easier to write code this way - sometimes you'll need to have small snippets of C code (such as function calls or small blocks of code) to augment your markups.

Comments


There are a few ways to comment your code while in a markup block.

Use either:

Outputting data

Whitespaces and empty lines


Semicolon


If semicolon is the last outputted character on the line, it must be escaped:
Hello world<?;?>
This is because any line that ends with semicolon is expected to be C code.

If you have lots of semicolons (for example in Javascript code), you can use start-verbatim/end-verbatim markups, in which case nothing in the output (including semicolon) is interpreted.

Encoding output


You can output data without any encoding, by web encoding or URL encoding:

Ignoring markup


Common outputting of HTML code is just to write it without any markup, as in this code:
Hello<hr/>
This is HTML<br/>
will produce:

Hello
This is HTML

If your output happens to coincide with a markup, such as for example, if it begins with if, then you could either output with a print-web (for example):
print-web "if here won't be treated as if markup"
or use w markup, which does nothing, but it positions whatever comes next as not being the first on the line, which precludes it from being interpreted as markup, for example:
w if here won't be treated as if markup
resulting in output:

if here won't be treated as if markup

If you have a large segment of text such as JavaScript that you just want to output to web page without worrying about any Cloudgizer markups being interpreted, or about trailing semicolons, use start-verbatim and end-verbatim:
start-verbatim
if here won't be treated as if markup<br/>
for here won't be treated as for markup<br/>
end-verbatim
resulting in output:

if here won't be treated as if markup
for here won't be treated as for markup

When start-verbatim is used, anything up to end-verbatim is output as if with print-noenc.

Outputting preprocessor data to a file

Use preprocessor-output# to output data during processing of source code files. For example:
preprocessor-output#indexes employee:name
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee where name='John'"
end-query
This will write to 'indexes.clo' file the line after the file name, in this case 'employee:name'. Such output can be used to determine what indexes are needed on each table used. In general any information can be added to any file. The purpose is to help with static analysis of code as in aforementioned example.

Outputting HTTP header for a typical web page


Before outputting any text to a web page, HTTP headers need to be sent out:
output-http-header
print-web "Hello world"
<hr/>
The header will contain any cookies that are present, meaning that had been received from the client, added and have not been deleted. Because this is the dynamic output from the program, the client is instructed not to cache.

If you want to send custom headers (for instance change content type, caching, status or any other options), you can use custom header API.

Defining, setting and outputting integers

Integers can be used via following markups:
// define integer with default value of 0
define-int new_int

set-int new_int=10

// define and set the value of integer with any expression
define-int my_int=4*new_int

set-int my_int=my_int+10
To output an integer:
print-int my_int
To output a long:
print-long my_long

Outputting data from a command line program

To output to standard output, use:
print-out "Today is a good %s", "day"
To output to standard error, use:
print-error "Today is a good %s", "day"
These constructs can be used only from a command line program. The syntax of the format and the data output is the same as for C printf() function.

Embedded C code

Use either:



With c, only one line of C code can be written that accompanies c.
With start-c...end-c, multiple lines of C code can be written. This C code is NOT placed in curly braces.

URL input parameters (GET and POST)

Input parameters from an incoming request are obtained by using input-param:
input-param par1
input-param par2

Input parameter par1 has value
print-web par1
, and input parameter par2 has value
print-web par2
This will create two string variables par1 and par2 and store input parameters par1 and par2 into them, respectively. For example, if URL that led here was:
https://mywebsite.com/go.your_app_name?par1=value1&par2=value2
then string variable par1 will have value "value1" and par2 will have value "value2", and the output from the above code is:

Input parameter par1 has value value1, and input parameter par2 has value value2

input-param works the same for both GET and POST requests. Input parameters are trimmed for whitespace (both on left and right).

Input parameter name can be made up of alphanumeric characters or underscore only and cannot start with a digit.

Cookies, setting and getting


Use cld_time API function to produce time suitable for cookies.

Database queries


You can SELECT, INSERT, UPDATE or DELETE database table, retrieve results, check for errors. You can not perform any other SQL statements with markups, for example DDL statements - if you want that, use the API.

You can connect to a single database, specified in .db file.

Writing queries, especially when many queries have common text, is supported with additional features such as query fragments and query shards.

Basic usage of queries



query-result# can be used with noencode (no encoding), urlencode (URL encoding) or webencode (web encoding), depending on where it used. For example, if it is used in construction of a <a href ..> link in the list of parameters, urlencode should be used. Default is webencode. These can be combined with define to store the value into it.
query-result#my_query,name define my_var urlencode
This will URL encode value of column 'name', then create string variable my_var and store the value into the string variable.

Database transactions


Once you start transaction with begin-transaction, you must either commit it with commit-transaction or rollback with rollback-transaction. If you do neither, your transaction will be rolled back once control is passed out of your request handler. Opening a transaction and leaving without committing or a rollback is a bug in your program.  If you want to catch this kind of situation and possibly either commit or rollback, use cld_check_transaction() API function (possibly within a function dispatcher in cld_handle_request(), after the request handling), where you can detect this and either rollback, commit, or error out.

Column Length


To output column length for a column in a table:
<input name="my_column" type="text" value="" size="30" maxlength="<?column-length employee.name?>">

// To get column column length into a string variable:
column-length employee.name as define my_column_length
// Or define a string variable first:
define-string my_column_length
column-length employee.name as my_column_length

<input name="my_column" type="text" value="" size="30" maxlength="<?print-web my_column_length?>">
The length obtained is the maximum number of characters in a column when it is output as a string. The supported database types for use in this markup are varchar, char and number types.

When query has no results

If your query returns no rows and you want still to get a single row made of empty strings for each column, use use-no-result:
define-query#my_query
use-no-result#my_query
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee where 1=2"
   query-result#my_query, name
   ,
   query-result#my_query, yearOfHire
end-query
This will produce output:

,

meaning empty name, then a comma, and then an empty year of hire.

Force query to produce a single empty row

Sometimes you may want to get a single row made of empty strings for each column, in which case use create-empty-row:
define-query#my_query
create-empty-row#my_query
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
   query-result#my_query, name
   ,
   query-result#my_query, yearOfHire
end-query
This will produce output:

,

meaning empty name, then a comma, and then an empty year of hire. Even though query would have produced some rows, you will still get a single empty row. The query is not actually executed to do this.

You may want to use this to conditionally either execute the query, or just produce an empty row, for example:
...
define-query#my_query
if execute_query!=1
   create-empty-row#my_query
end-if
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
   query-result#my_query, name
   ,
   query-result#my_query, yearOfHire
end-query
In this case, the same code is used to either show the data from the database or an empty row, such as if you wanted to present a blank form to be filled out.

Current row in a query result set

Use current-row# to get the current row number, starting with 1.
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
   Row #<?current-row#my_query?><br/>
   query-result#my_query, name
   ,
   query-result#my_query, yearOfHire
   <br/>
end-query
This produces output:
Row #1
Linda,2001
Row #2
John,2003

You can also store current row in a string variable first:
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
   // You can define variable on the spot
   current-row#my_query as define curr_row
   // Or you can define it first:
   define-string curr_row
   current-row#my_query as curr_row

   Row #<?print-web curr_row?><br/>
   query-result#my_query, name
   ,
   query-result#my_query, yearOfHire
   <br/>
end-query
The output produced is the same.

Number of rows and columns in a query result set

Use row-count# and column-count# to get the number of rows and columns returned by the query.
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
   Data is <?query-result#my_query, name?>, <?query-result#my_query, yearOfHire?> (row <?current-row#my_query?> out of <?row-count#my_query?> with <?column-count#my_query?> columns) <br/>
end-query
Number of rows: <?row-count#my_query?><br/>
Number of columns: <?column-count#my_query?><br/>
This produces output:
Data is Linda, 2001 (row 1 out of 2 with 2 columns)
Data is John, 2003 (row 2 out of 2 with 2 columns)
Number of rows: 2
Number of columns: 2

You can also store number of rows and columns in a string variable:
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
end-query
// Define variable on the spot
row-count#my_query as define row_count
column-count#my_query as define col_count
// Or define a string variable first
define-string row_count
define-string col_count
row-count#my_query as row_count
column-count#my_query as col_count

if atol(row_count)!=2
   <div>Number of rows is not 2, but rather it is <?print-web row_count?>!</div>
end-if
If number of rows in the table is 3, the following output is produced:

Number of rows is not 2, but rather it is 3!

Note that row-count and column-count can be used within run-query loop and also right outside of it as well.

Number of affected rows in DML

Use query-result#query_name,affected_rows to get the number of rows affected in INSERT, UPDATE or DELETE.
run-query#my_query="insert into employee (name, dateOfHire) values ('Terry', now())"
   Number of rows inserted is <?query-result#my_query,affected_rows?>
   // You can also get number of rows into a string variable
   query-result#my_query,affected_rows as define nrows
   if-string nrows!="1"
       report-error "Cannot insert data!"
   end-if
end-query
In the above example, we check the number of rows affected (in this case inserted). Consult database manual for more information about number of rows affected - it may not always be what you'd expect.

Input parameters to queries


Query input parameters are specified via <?...?> markup. Do not confuse query input parameters (data used in SQL statements) with web input parameters (which are name/value pairs from input URL specified with input-param).
input-param employee_name
input-param year_of_hire

run-query#my_query="insert into employee (name, yearOfHire) values (<?employee_name?>, <?year_of_hire?>)"
   New employee ID is <?query-result#my_query,insert_id?>
end-query
In the above example, web input parameters employee_name and year_of_hire (specified in the URL such as yourserver/go.your_application?employee_name=...&year_of_hire=...) are used as query input parameters in INSERT statements.

Query input parameters are always strings (i.e. char*) by type and can be any C expression, for example:
run-query#my_query="insert into employee (name, yearOfHire) values (<?by_employee==1 ? entity_name : employee_name?>, <?calculateYearOfHire() + 1?>)"
In the above example, both employee name and year of hire are results of C expressions.

Another example:
run-query#my_query="select yearOfHire from employee where name=<?employee_name?>"
   query-result#my_query,yearOfHire as define year_of_hire
   Employee found, hired in year: <?print-web year_of_hire?>
end-query
Query input parameters are sanitized against SQL injection attacks, and they are also trimmed on both left and right if trim-query-input is in effect.

To trim all query input parameters, use:
trim-query-input
To not trim, use:
no-trim-query-input
Any query occurring at run-time after either markup will be under its effect. By default, all query input parameters are not trimmed.

Column names

You can obtain the names of query columns by using column-names# markup:
run-query#some_query="select * from employee"
   column-count#some_query as define col_count
   column-names#some_query as define col_names
   define-int i
   for i = 0; i < atoi(col_count); i++
       Column #<?print-web i?> has name <?print-web col_names[i]?>)
   end-for
end-query
Note that column-names# means the names of query columns, which may or may not match any actual table column names, since query outputs can have aliases (and they should have them if the output is computed). In the following example, the output will be 'employeeFirstName' and 'employeeLastName' as they are aliases:
run-query#some_query="select firstName employeeFirstName, lastName employeeLastName from employee"
   column-names#some_query as define col_names
    Column names are <?print-web col_names[0]?>  and <?print-web col_names[1]?>
   end-for
end-query
column-names# must have as clause. In the above example, the variable 'col_names' is defined automatically, but you can also define it yourself and omit the define clause:
c char **col_names;
column-names#some_query as col_names
Note that column-names can be used within a query loop (like we did above, such as within run-query/end-query for example), or just outside of it.

Nesting queries


Queries can be nested, for example:
run-query#query1="select id from user"
   query-result#query1,id as define id
   run-query#query2="select timeout from settings where id=<?id?>"
       query-result#query2,timeout
   end-query
end-query
In this example, query2 is nested within query1, using its results. Note that query-result# for a query can be used only directly within a query loop, and not within another query's loop. For instance, query-result#query1,id cannot be used within a query loop for query2. This restriction is solely in place to prevent errors that are hard to track, such as having same column names under different queries. It is always possible to use query results directly within the loop, and is a better programming practice.

Using multiple query texts at run-time (conditional queries)

A query can have different text depending on run-time conditions. The text of the query is still static though:
input-param my_name_filter
input-param my_action


define-query#my_query

// depending on a run-time conditional, start two different queries
if-string my_action="show_all_records"
   start-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
else
   start-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee where name like concat(<?my_name_filter?>,'%')"
end-if

// loop through a query, regardless of which one we will execute in loop-query
loop-query#my_query
   Name is <?query-result#my_query, name?>, year of hire is <?query-result#my_query, yearOfHire?><br/>
end-query
If the input parameter for my_action is 'show_all_records' (for example URL is https://mysite.com/go.your_app_name?my_action=show_all_records), the output is:

Name is Linda, year of hire is 2001
Name is John, year of hire is 2003

If the input parameter for my_action is empty and input parameter for my_name_filter is 'L' (for example URL is https://mysite.com/go.your_app_name?my_name_filter=L), the output is:

Name is Linda, year of hire is 2001

For conditional queries with the same name, even though the query texts (the SQL texts) can be different and they can have different input parameters, they must have the same output, because the results sets of all such query texts are used in a single loop-query loop. For example, in the above conditional query my_query, one query text has no input parameters, and the other one has a single input parameter. However, both  texts have the exact same output ('name' and 'yearOfHire').

Dynamic queries

A dynamic query has text that is not known at compile time, i.e. it is produced at run-time. Dynamic query cannot be checked at compile time. Dynamic query should be used rarely and only when absolutely necessary. The following is an example and would be normally written as a conditional query.
// Get input parameters
input-param employee_name
input-param employee_year

// The base query that's present either way
define-string query_text="select name, extract(year from dateOfHire) from employee where dateOfHire>'%s'-01-01 0:0:0' "

// State that dynamic query comes from string variable query_text and specify what are output column names
define-dynamic-query#my_query=query_text with-output name, yearOfHire

// Query text that's added to the base query if employee_name is specified
define-string where_clause=" and name='%s'"

// Construct the query text at run-time. Add appropriate parameters.
if-string employee_name!=""
   append-string where_clause to query_text
   // We have two input parameters in this case
   add-query-input#my_query: employee_year, employee_name
else
   // We have one input parameter in this case
   add-query-input#my_query: employee_year
end-if

// Execute query. Only at this point do we actually use the dynamic text of the query.
run-query#my_query
   Employee <?query-result#my_query, name?> was hired in year <?query-result#my_query,yearOfHire?><br/>
end-query
The text of query is a variable query_text. This variable is constructed at run-time based on whether input parameter employee_name is provided or not. Clause with-output is used to specify what are the names of output columns - their order and names must match the query's output columns.

Since the query is dynamic, the query's input variables are not known and must be specified. Input variable is specified with '%s'. The actual input variables are provided in add-query-input# clause.

If the input parameter for employee_name is Linda (for example URL is https://mysite.com/go.your_app_name?employee_year=1990&employee_name=Linda), the output is:

Employee Linda was hired in year 2001

If the input parameter for employee_name is empty (for example URL is https://mysite.com/go.your_app_name?employee_year=1990&employee_name=   or just    https://mysite.com/go.your_app_name?employee_year=1990), the output is:

Employee Linda was hired in year 2001
Employee John was hired in year 2003

Dynamic queries when table structure is unknown


In some cases, you would not know the output columns of a dynamic query. For example, a dynamic query could be constructed in form of 'SELECT * from ...'. In general it may be beneficial in certain situations to obtain query results even if you don't know (or is cumbersome to obtain) the query output columns. In these cases you can use with-unknown-output markup, as in this example:
// Get all tables from current database
run-query#all_tabs="select table_name from information_schema.tables where table_schema=database()"
   query-result#all_tabs, table_name as define table_name

   // Construct the run-time text of a dynamic query
   write-string define qry_txt
   select * from <?print-noenc table_name?>
   end-write-string

   // Define dynamic query to be the text above and specify that we don't know column names
   define-dynamic-query#get_tab = qry_txt with-unknown-output

   // Run query
   run-query#get_tab
   end-query

   // Get some table metadata and also the actual table data
   row-count#get_tab as define row_count
   column-count#get_tab as define col_count
   column-names#get_tab as define col_names
   column-data#get_tab as define col_data

   // Display data
   define-int i
   define-int j
   for j = 0; j <atol(row_count); j++
       // Display columns for each row
       for i = 0; i <atol(col_count); i++
           // Show column names and column data (which is a flat array that spans all column and all rows)
           print-out "colname %s, coldata %s\n", col_names[i], col_data[j*atol(col_count)+i]
       end-for
   end-for
end-query
In this example, we don't know table names (or their columns) because we get the list of tables from the current database. Query text is constructed on the fly and we define dynamic query with-unknown-output because we don't know the output column names. After running the query we obtain the number of rows (row-count#), the number of columns (column-count#), the output column names (column-names#) and the actual data (column-data#).

The example above is typical for applications that export data or move it around while massaging the data in some ways.

Result-set data array


Normally you'd obtain query results using query-result# as in:
run-query#some_query="select firstName, lastName from employee"
   query-result#some_query, firstName as define first_name
   query-result#some_query, lastName as define last_name
   Employee (
   print-web first_name
   ,
   print-web last_name
   ) found<br/>
end-query
In some cases, it may be better to not refer to result data set in this fashion, meaning by column name. For instance, in a dynamic query you may not know the table name, nor column names. In these cases, you can use column-data# markup:
void get_table_data (const char *table_name)
{
   /*<
   //
   // Construct the run-time text of dynamic SQL
   //
   write-string define qry_txt
   select * from <?print-noenc table_name?>
   end-write-string

   //
   // Connect dynamic query with the run-time text of SQL
   // We use with-unknown-output to demonstrate a solution if we don't know
   // (or is difficult) to obtain output columns of a query
   //
   define-dynamic-query#get_tab = qry_txt with-unknown-output

   //
   // Run the dynamic query
   //
   run-query#get_tab
   end-query

   //
   // Obtain number of rows, number of columns, column names, the actual column data
   //
   row-count#get_tab as define row_count
   column-count#get_tab as define col_count
   column-names#get_tab as define col_names

   //
   // Get actual table data
   //
   column-data#get_tab as define col_data

   
   //
   // In a loop, go through all rows, and for each row, display all column info as well
   // as the actual column data from the table.
   //
   define-int i
   define-int j
   for j = 0; j <atol(row_count); j++
       //
       // Display columns for each row
       //
       for i = 0; i <atol(col_count); i++
           print-out "colname %s, coldata %s\n", col_names[i], col_data[j*atol(col_count)+i]
       end-for
   end-for
   >*/
}
In the above, a C function get_table_data prints out (to stdout) a list of columns, and a column value (note that this would have to be a command-line program since that's where we'd use print-out).

column-data lets us obtain all table's data in a char**, i.e. in an array of strings, where all table's data is laid out in a single data array, organized by repeating rows. For instance, suppose that table name in our example has 2 columns. In that case, col_data[0] and col_data[1] would be the two columns' values from the first row, col_data[2] and col_data[3] would be the two columns' values from the second row, col_data[4] and col_data[5] would be the the columns' values from the third row etc.

column-data# must be used with as - in other words a suitable char** variable must be provided, either defined with as define or provided as in:
c char **col_data;
column-data#get_tab as col_data

Using query results more than once


A query result set can be looped through again:
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
   query-result#my_query, name <br/>
end-query

loop-query#my_query
   Again: query-result#my_query, name <br/>
end-query

loop-query# will go through the entire result set again, producing this output:

Linda
John
Linda
John

Note that loop-query# does not execute a query again, but rather only uses the already obtained results.

Copying query text in multiple places safely (query shards)



Re-using query text in multiple places (query fragments)


To reuse the text of query segment, use query-fragment#:
input-param find_name
input-param find_year

query-fragment#my_fragment="select name, extract(year from dateOfHire) yearOfHire from employee "

if-string find_name!=""
   run-query#my_query_one="<?query-fragment#my_fragment?> where name=<?find_name?>"
else-if-string find_year!=""
   run-query#my_query_one="<?query-fragment#my_fragment?> where dateOfHire > '<?find_year?>-01-01 0:0:0' "
end-if
Query fragment is a string constant that can be used within a query text. It is useful where shards are not practical, such as if a query fragment is too long, or used in too many places. Each query is still static because the text is known at compile time. A fragment can have other constructs in it, such as input parameters, other fragments or shards.

Loop control in result set


When looping through the result set, you can use continue-query and exit-query to continue at the top of the loop or exit the loop, respectively:

This will print only the first employee name:
c int i = 0;
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
   query-result#my_query, name
   c i++;
   if i>0
       exit-query
   end-if
end-query
This will skip the first employee name and print the rest:
c int i = 0;
run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
   c i++;
   if i==1
       continue-query
   end-if
   query-result#my_query, name
end-query

Error handling in executing queries


For INSERT, DELETE and UPDATE statements, error value is available in the result set:
run-query#my_query="insert into employee (name, yearOfHire) values ('Terry', 2017)"
   query-result#my_query, error as define err
   if atol(err)!=0
       report-error "Error in adding employee, error %s", err
   end-if
end-query
SELECT statements must succeed (with or without results). If not, it is an irrecoverable error.

Getting auto_increment value


To obtain auto_increment value after insert, use insert_id value:
run-query#my_query="insert into employee (name, yearOfHire) values ('Terry', 2017)"
   query-result#my_query, insert_id as define employee_id
   if atol(err)!=0
       report-error "Error in adding employee, error %s", err
   end-if
   New employee added, with an ID of
   print-web employee_id
   <br/>
end-query
Just like with most other markups, you can define the string as you're getting it, or use the string you already defined:
   query-result#my_query, insert_id as define employee_id

   // or:

   define-string employee_id
   query-result#my_query, insert_id as employee_id

NULL and result column data types


All results from the database come as a string type (i.e. char*). NULL value is always an empty string (""). If you want to know if a value is NULL (versus an actual empty value), use a function such as ISNULL() in the database queries.

Executing Cloudgizer program from command line

See this chapter in API on more information about building and executing Cloudgizer program from command line.

Executing external programs

For most use cases, use exec-program:
define-int st
define-int out_var_len=2000
define-string out_var
define-string program_path="/usr/bin/some_program"

exec-program program_path program-args "-flag1", arg1, "-flag2" program-status st program-output out_var program-output-length out_var_len
This will execute program specified by program_path with arguments specified under program-args. Program arguments are separated by a comma. If a comma is a part of an argument, it must be escaped, as in \, and if double quote is a part of an argument it should be escaped as in \" as well.

On return, the exit status will be st, and any output (stdout and stderr) will be in out_var. Note that out_var is allocated inside the execution function, and the length used to allocate it is out_var_len.

If program-output-length is not specified, the out_var output variable is allocated with a default 256 bytes. Either way, any output in excess of what's available is truncated.

You can also define the status of execution and the output buffer in the markup itself:
exec-program "/usr/bin/some_prog" program-args "-flag1", arg1, "-flag2" program-status define st program-output define out_var
If you want more complex use cases, such as redirecting file to stdin, and redirecting stdout/stderr to a file, or if you want to create a "pipe", i.e. execute two programs where output of one is the input of another use program execution functions.

Loading URL content (calling a web page) programmatically

Use web-call to get the result of loading the content of a URL via GET method:
web-call "https://mysite.com/any-url" with-response define webres with-error define weberr  with-cert "/home/zigguro/ca.crt" cookie-jar "/home/zigguro/cookies"
A web response to a given URL is loaded into a string variable that is defined with a mandatory with-response clause. In this case variable webres is defined via define clause, but it could be defined prior to web-call. Other clauses are not mandatory. with-error specifies a string variable to hold an error text, if any. with-cert is used to specify the location of HTTPS certificate. You can also use with-no-cert if the certificate is not to be checked.

cookie-jar specifies the location of a file holding cookies. Cookies are read from this file (which can be empty or non-existent to begin with) before making a web-call and any changes to cookies are reflected in this file after the call. This way, multiple calls to the same server maintain cookies the same way browser would do. Make sure the same cookie-jar file is not used across different application spaces.

A simple use of web-call:
define-string webres
web-call "http://mysite.com/any-url" with-response webres
or without checking HTTPS certificate:
define-string weberr
web-call "https://mysite.com/any-url" with-response define webres with-error weberr  with-no-cert
Response that redirects to another URL will be followed up to 4 times. The response is the body message, not the headers.

This function does not return length of a response, because it is meant to be used as a text-based (i.e. null-terminated string based) messaging system. For uploading or downloading files, a more encapsulated method would be to use command-line utilities (such as curl or rcopy) via program execution functions.

Strings


Strings are 'char *' variables that are allocated by Cloudgizer's memory allocation system. Typically you never free them. They are freed at the end of each request.

If you are using any cld_* function that alters strings (i.e. as an output parameter) or in markup constructs that alter strings (such as append-string or write-string), then such strings must be defined using one of the following methods, or by means of API functions such as cld_malloc, cld_calloc or cld_realloc. Usage of other kinds of 'char *' variables will likely be detected by Cloudgizer and program halted.

Defining and copying strings


You should not assign NULL value to a string. Use empty strings (as described above) instead.

Appending string

Use append-string:
define-string str1="Hello "
define-string str2="world!"

append-string str2 to str1

print-web str1
The output is:

Hello world!

Search and replace string

Use subst-string and subst-string-all:
define-string my_str = "Have a good day and good night!"
subst-string "good" with "great" in my_str

print-web my_str
In the above example, the very first occurance of 'good' is replaced with 'great', producing the output:

Have a great day and good night!

If you wish to replace all occurances, use subst-string-all:
define-string my_str = "Have a good day and good night!"
subst-string-all "good" with "great" in my_str

print-web my_str
The above will produce the output:

Have a great day and great night!

The string where replacement takes place (in this case 'my_str') must have been defined with define-string.

Constructing complex strings

Instead of output being sent to the Web client, it can be directed into a string.  Use write-string and end-write-string to do this:


All leading and trailing whitespaces are trimmed from each line. Only the whitespaces within each line are output. Text is output as non-encoded (as opposed to URL or web encoded), unless you specifically output something encoded otherwise.

Program flow

Loops

Use for and end-for markups:
define-int it
for it=0; it < 10; it++
   Outputting #<?print-int it?><br/>
end-for
In the above example, a simple loop outputs this to the web page:

Outputting #0
Outputting #1
Outputting #2
Outputting #3
Outputting #4
Outputting #5
Outputting #6
Outputting #7
Outputting #8
Outputting #9

Conditional statements (if, else...)

Conditional statements can be used as follows:

Files

File storage


You can store your files anywhere you like and use either standard C file mechanism to handle them or Cloudgizer API (such as reading/writing files etc.). Cloudgizer however, also provides a file storage mechanism that you can use for any general purpose. This file storage is used for automatic upload of files as well.

In this file storage, files are generally stored in file directory under application's home directory. Your application may use this storage for any documents you create, see the API you can use to create your own files. This storage is also used for uploading of files from web clients (such as uploading files from web browsers or mobile device cameras) - the uploading is handled automatically by Cloudgizer.

About 30,000 files will be stored in a single subdirectory (i.e. /home/user/file/d<number>) before moving to the next subdirectory. This number of files may increase in the future, however that will not affect files in existing installations, i.e. they will stay where they are.

The number of actual subdirectories and files is in theory limited only by the filesystem used, and Cloudgizer's choice of the number of files is not a hard limit, hence it may increase in the future. The number chosen currently is reflective of the common device storage available at present and the subjective typical file size distribution across typical business applications, so clearly the choice is not related to theoretical limits or considerations.

Modern filesystems can support great many files. For example, if you had 120 million files, they would be spread across some 40,000 directories with 30,000 files each. For example, files might be named /home/user/file/d0/f31881, /home/user/file/d10/f321214, etc.

File names are always based on the ID number, so that changing the number of files per subdirectory will never overwrite files. This scheme also allows for faster access to file nodes due to relatively low number of files per subdirectory, and also might allow for hardware or network partitioning.

Typically, an application will store file names in the database and actual files in file storage in file directory. Each file has a unique ID, and you can use this ID to reference a file in the database, see an example in file creation API.

The table cldDocumentIDGenerator is used to create unique file names across the application, even when multiple requests are made at the same time. This table is created as part of Cloudgizer's installation and used internally to deliver this functionality - you don't have to ever deal with it.

Permissions and ownership


Any files created by Cloudgizer (i.e. through uploading from web clients, writing files etc.) are owned by Cloudgizer user and have 700 access permission (owner-only reading, writing and accessing). Group has no privileges, and neither have other users. The Cloudgizer user's home directory is also set up with 700 access permission.

Uploading files


Cloudgizer will upload files for you automatically. The file will be stored in file directory by using the document ID generator as a basis for subdirectory and file name under the file directory.

For example, files uploaded might be named /home/user/file/d0/f31881, /home/user/file/d10/f321214, etc. See File storage for more details on how files are stored.

Reading a whole file

Use read-file to read a whole file:
read-file "/home/user/some_file" to define file_content status define st
if st>=0
   Read:
   <hr/>
   print-web file_content
   <hr/>
else
   Could not read (<?print-int st?>)
end-if
The above example will read a file /home/user/some_file and store the result into a variable file_content and the status into variable st.  Variable st would be -1 if cannot open the file, -2 if cannot read the file, and the size of the file read (0, or a positive number) if reading succeeded.  

define in both the content variable and the status are optional, so it could be:
define-int st
define-string file_content

read-file "/home/user/some_file" to file_content status st

Writing a whole file or appending to a file

Use write-file to write to a file:


Copying a file

Use copy-file to copy a file to another file. The target file is created if it does not exist.
copy-file "/home/user/source_file" to "/home/user/target_file" status define st
print-int st
This copies from file source_file to file target_file. The result is -1 if cannot open source_file, -2 if cannot open target_file, -3 if cannot read source_file, -4 if cannot write target_file, and 1 on success.

You can omit define for status:
define-int st

copy-file "/home/user/source_file" to "/home/user/target_file" status st

Web address of server

Web address is the URL specified in config file in web_address variable.

Use web-address to get the base URL for server. Use it for links and forms:


Error handling in general


If an irrecoverable error occurs, then oops function will be called. Irrecoverable error is an error that cannot be handled, it always means the end of a request. Examples of such errors are invalid SQL statements or an invalid URL request. oops function is like this:
#include "cld.h"
..
void oops(input_req *req, const char *err);
input_req is a type that contains information relating to the current request. err is an error message itself.

You can change the implementation of oops function from what it is by default.

When checking errors in your application (i.e. errors that are recoverable), and if upon the examination of the error, your code decides it is still unrecoverable, do not call oops directly. Rather use report-error:
if cannot_recover==1
   report-error "Error in processing, error %s, details %s", err, details
end-if
report-error takes arguments the same way as standard printf function.  

Include files


Do not use Cloudgizer markup code in include files. Instead place all of it in .v files, which are translated into .c files. Use inline code if you need to avoid function calls.

Debugging your code

Generally, you can debug Apache server by forcing it into a single-process mode, running it in gdb and hunting for problems there. A good tutorial on this can be found at https://httpd.apache.org/dev/debugging.html. Once in debugger, you'd want to break in cld_main function, which is the root function for any Cloudgizer application. If for some reason this doesn't work, you have another option, described here.

Tracing and debugging options

These options are located in the trace/debug file, meaning debug file in the trace directory under application's home directory. Typically, you'd have this in it during development:
sleep=-1
trace=1
lint=1
memorycheck=1
tag=<anything you like>
Here is the breakdown of what these do:


Make sure not to use sleep, lint, memorycheck and tag in your production code.

Finding where program crashed


When your program crashes, your best bets are to look in:


Both web-page-crash and backtrace file will attempt to show the full stack dump, and there you can see function names and line numbers (if available and if you compiled with debugging information included). Cloudgizer will analyze your executable and attempt to show the exact line in your source where the problem happened (meaning the exact line in .v code, and not in the generated .c code, alleviating the need to connect the two files!).

For example, you'd see a report like this that exactly pinpoints the location of the crash (in this case due to segmentation fault) - you'd look for the source code just before entering signal_handler and you'll also see the entire stack leading up to invocation of the function where the problem is:
START STACK DUMP ***********
28014: 2018-06-01-20-54-04: Caught SIGSEGV: segmentation fault
last known tracing file/line: [cldrt.c][2697]
-----
cld_get_stack
/home/hellocld/src/chandle.c:165
/usr/local/lib/cld/libacld.so(cld_get_stack+0x50) [0x7f4cd747016c]
-----
posix_print_stack_trace
/home/hellocld/src/chandle.c:136
/usr/local/lib/cld/libacld.so(posix_print_stack_trace+0x10) [0x7f4cd7470101]
-----
signal_handler
/home/hellocld/src/chandle.c:264
/usr/local/lib/cld/libacld.so(signal_handler+0x1f0) [0x7f4cd7470504]
-----
__restore_rt
sigaction.c:?
/lib64/libpthread.so.0(+0xf370) [0x7f4cdd086370]
-----
home_landlord
/home/hellocld/rentomy/home_landlord.v:199
/usr/lib64/httpd/modules/libcldapp_rentomy.so(+0xab200) [0x7f4cd7752200]
-----
home
/home/hellocld/rentomy/home.v:155
/usr/lib64/httpd/modules/libcldapp_rentomy.so(+0xa05f2) [0x7f4cd77475f2]
-----
pass_reset
/home/hellocld/rentomy/pass_reset.v:359
/usr/lib64/httpd/modules/libcldapp_rentomy.so(+0x9ed87) [0x7f4cd7745d87]
-----
cld_handle_request
/home/hellocld/rentomy/cld_handle_request.v:291
/usr/lib64/httpd/modules/libcldapp_rentomy.so(+0x57e6) [0x7f4cd76ac7e6]
-----
cld_main
/home/hellocld/rentomy/a_cldapp.c:69
/usr/lib64/httpd/modules/libcldapp_rentomy.so(+0x4248) [0x7f4cd76ab248]
-----
cld_handler
/home/hellocld/rentomy/mod.c:52
/usr/lib64/httpd/modules/libcldapp_rentomy.so(+0xad4d4) [0x7f4cd77544d4]

Enabling debugging information

To enable debugging with gdb, set CLDDEBUG variable in supplied cldmakefile to 1 when making your application. This will turn on debug code generation for gdb.

Which shared libraries are loaded?


Check the beginning of the request's trace file - it shows all shared objects loaded, their names and their address ranges. This is useful when figuring out the configuration of Apache server, and otherwise.

API


The following is the list of publicly available data types and functions.

Do not use API for common markups provided otherwise. For example, do not use input parameters API, but rather use input-param markup. If underlying API and types change, you'll have to change your code.

Use API only if there is no alternative. The same goes for any other C code you may write, be in direct C code or through c and start-c markups.

Memory allocation

Allocation and garbage collection

The following memory allocation functions are available:
void *cld_malloc (size_t size);
void *cld_calloc (size_t nmemb, size_t size);
void *cld_realloc (void *ptr, size_t size);
void cld_free (void *ptr);
char *cld_strdup (const char *s);
The above have the same interface as the standard library functions without the 'cld_' prefix.

Always use define-string to allocate memory - only if otherwise necessary, use above cld_* functions for memory allocation.

Most all markups and API will automatically create or expand/shrink memory when needed.

Never use cld_free unless absolutely needed, for example if your request allocates lots of memory in a loop and needs to release it in each pass during a single request.

Memory garbage collector is provided and will release all allocated memory at the end of request.

Do not try to minimize memory usage of a request unless you have a very good reason to, because most requests take relatively short time to complete and memory will be released shortly. Great many bugs and crashes come out of using free()-like functions improperly.

Never use library functions malloc(), free() etc. unless you have an extremely compelling reason, for example if you need to allocate memory and some library frees it. Do not use pointers obtained from Cloudgizer with library functions such as free() or realloc(), and vice versa.

Get size of memory allocated for variable

When variable is allocated by means of define-string or cld_malloc (or such) functions, you can determine at run-time how many bytes are currently allocated for this variable:
int sz;
char *ptr = cld_malloc (200);
cld_check_memory (ptr, &sz);
Variable 'sz' will have value of somewhat over 200 (accounting for overhead in allocating memory). If pointer 'ptr' was not allocated by Cloudgizer functions or markup, program will error out.

Check if allocated memory is valid, overwritten or underwritten

If you want to check if memory pointer is valid and if memory hasn't been overwritten or underwritten, use cld_check_memory:
int i = cld_check_memory (p, NULL);
p is a pointer returned from define-string, cld_malloc or other such functions. This is an assertive function, i.e. your program will report an error if the memory block is invalid. If memory is okay, it returns an index into an internal memory array that holds information about allocated pointers. This index is for internal purposes only, do not use it in any way.

Configuration parameters


Configuration parameters are filled from config file, located in the application's home directory.

Use members of cld_get_config()->app structure to get the following:


Tracing and debugging

Write to trace file


Trace call allows you to write to current process' trace file:
CLD_TRACE("Entering new employee, id %s", id);)
Trace call uses printf syntax to output data.  

Trace will be written only if trace parameter in debug file is 1.

Which trace file is being written to?


The name of current trace file is:
cld_get_config()->app->trace.fname

Using custom tag value for debugging


Use cld_get_config()->debug.tag to access tag field in the debug file. This field can have any value you want. Use it to control some aspect of debugging process, in a conditional fashion:
if-string cld_get_config()->debug.tag = "$debug_check$"
   ...
end-if
In production, you would clear the tag value.  

Linting your HTML code dynamically

Use cld_lint_text to check validity of HTML code generated dynamically at run-time (for example check HTML emails before sending or check snippets of generated code not directly displayed):
write-string define html_code
   <!DOCTYPE html>
       <html>
           <body>
                Hello<br/>
                This is html code.<br/>
                <hr/>
           </body>
       </html>
end-write-string

c cld_lint_text (html_code);
In this example, Cloudgizer will check validity of HTML in html_code (as XHTML) by using xmllint utility. lint in debug file must be set to 1 in order for cld_lint_text to do this. If lint is 0, nothing will be done.

If there is an error, program will stop and you can see more information in program's trace file (under trace directory).

Note that Cloudgizer will lint your web output automatically if lint is 1. Use this function to check snippets of HTML that are not output directly by your program.

URL input

URL input parameters


Input parameters are stored in
cld_get_config()->ctx.req->ip
Total number of input parameters is:
cld_get_config()->ctx.req->ip.num_of_input_params
Names and values of input parameters are:
cld_get_config()->ctx.req->ip.names[0], cld_get_config()->ctx.req->ip.values[0]
cld_get_config()->ctx.req->ip.names[1], cld_get_config()->ctx.req->ip.values[1]
...
etc. up to the total number of input parameters

Is URL referrer from the same site?


Use cld_get_config()->ctx.req->from_here to determine if the current web request originated from the same site or not. For example, you may not want to spend bandwidth on images not shown from your application, if that's how it's designed. This flag is 1 if the current web request originated from the same site, otherwise 0.

Note that from_here will be 1 if the URL is opened from a web client directly by the end-user.

URL string that is being processed


The URL of current request is:
cld_get_config()->ctx.req->url
and the referring URL is:
cld_get_config()->ctx.req->referring_url

URL parameters manipulation


There is a cld_input_params data type that can hold input parameters. You can manipulate URL parameters directly by using cld_replace_input_param function with cld_get_config()->ctx.req->ip as input.

You can use the following:


Cookies

Data type for storing

Cookies passed from HTTP are in:
cld_get_config()->ctx.req->cookies[0].data
cld_get_config()->ctx.req->cookies[1].data
....
up to the number of cookies, which is:
cld_get_config()->ctx.req->num_of_cookies

Cookies: get, set, delete, find



Global data


If you need global data in your program (data accessible to any function) and share it between different parts of your program, you can use:
cld_get_config()->ctx.req->data
This is a global void* pointer you can use any way you like for sharing global data unrelated to the current request data.

Encoding and Decoding, URL encoding, Web encoding, Base64 encoding



Numbers

Create string from integer


To get a new string from an integer, use cld_i2s function:
print-web cld_i2s (55, NULL);
The result will be:

55

To store the value in a string:
define-string val

c cld_i2s (55, &val);

print-web val
The result is the same.  

Check if string is a positive integer or a number



Execution of external programs, piping from one program to another


You can execute external programs by using several functions. All of these functions return the status of program (typically 0 if okay, non-zero otherwise).

Typical use cases are:



Here are the details on these functions:


Output data and HTTP header

Outputting HTTP header, custom HTTP response headers


To output HTTP header for dynamically generated web pages, use cld_output_http_header:
input_req *req = cld_get_config()->ctx.req;

cld_output_http_header(req);
In most cases, you'd just output header as shown above. This means a client (such as browser) is instructed not to cache since pages are dynamically generated in a typical scenario and content type is HTML.

If you however want to send a custom header, you can set it through header member of input_req:
input_req *req = cld_get_config()->ctx.req;
cld_header header;

header.ctype = "..."; // set content type such as "text/css"
header.cache_control = "..."; // set custom cache control, such as "max-age=10000"
header.status_id = "..."; // set custom response code such as "302"
header.status_text = "..."; // set custom response text such as "Found"
// Set any other custom headers as name/value pairs in control/value array, which must not have gaps and unused values must be NULL
// There can be maximum of CLD_MAX_HTTP_HEADER headers added through this mechanism.
header.control[0] = "...";
header.value[0] = "...";
header.control[1] = "...";
header.value[1] = "...";
...

req->header = &header;
cld_output_http_header(req);
In the above example, custom header is constructed and output.

Encoding data for output


To output data, use cld_printf or cld_puts.



Each of the above functions returns number of bytes written. Writing is typically buffered for maximum performance.

"Outputting data" means outputting to the web client, or the string. If the above functions are used within write-string and end-write-string (or API equivalents), the output goes to the string, otherwise it goes to the web client, except if the program is in batch mode. In batch mode, there is no output produced by these functions - you have to output data yourself (to a file, stdout, stderr etc.) or by using print-out and print-error markups. In batch mode, you can still write to a string and then output the string however you see fit.  

Output text without breaks


If you want to output text to web client and substitute new lines with <br/>s, use:
const char *text = "this is\nsome text\n";
cld_print_web_show_newline (text);
This outputs:

this is<br/>
some text<br/>

Disable and Enable HTML output, status of enablement


The output of text to web client can be enabled or disabled. By default it is enabled. In order to output a file, it has to be disabled.

To disable output:
cld_disable_output();
To enable output:
cld_enable_output();
You would disable HTML output when you want to output a file from your program, in which case you would output HTTP headers on your own (see cld_out_file()).

When HTML output is disabled, anything written so far into the web output buffer (that hasn't been output yet) is cleared. Also, anything output from that point onward is never written to the output buffer.

If you need to know if html output is enabled or disabled, use cld_is_disabled_output() function. It returns 1 if HTML output is disabled, otherwise 0.

Send files to the client and output HTTP headers for files


Use cld_out_file to send files to the client, for example images, documents etc. You can do this to display these files or to have the client download them.

Because the output from your program is always generated text, it is always sent to client with a note not to cache (unless you use custom headers).

For outputting files, however, you can control HTTP headers. cld_header data type can be used to control it:
cld_header header;

//
// Set the content type
//
header.ctype = "image/jpg"; // (or text/html or application/pdf etc.)
Content type must be specified.
//
// Disposition, show or download
//

// Show file
header.disp = NULL; // default, just show in client
header.file_name = NULL; // default, just show in client, no file name to download

// Download file
header.disp = "attachment"; // used to download files (i.e. client will prompt for download)
header.file_name = "document14"; // this is file name to be downloaded, used only when disp is "attachment"
Default values disp and file_name are  NULL.
//
// Cache control
//
header.cache_control = "max-age: 3600"; // set cache-control.
Default cache value is to cache almost forever (or rather for 53 years).
//
// ETag
//
header.etag = 1; // if 1, etag (file modification date) will be included, if 0, it won't be
Default etag value is 1.
//
// Custom HTTP header fields
//
header.control[0] = "some_HTTP_option";
header.value[0] = "value for some_HTTP_option";
header.control[1] = "some_HTTP_option_1";
header.value[1] = "value for some_HTTP_option_1";
etc. up to CLD_MAX_HTTP_HEADER options. All options in control/value members must be continuosly present from index 0 up to whatever index is used.
By default, all custom fields are NULL.

An example:
cld_header header;

// Initialize header
cld_init_header (&header);

// Set header
header.ctype = "image/jpg";

// Output file with header
cld_out_file ("/home/user/myfile.jpg", &header);
Both cld_init_header and cld_out_file do not return a value.

Note that cld_out_file will error out if your file name has dot-dot (..) in it - such files will not be served to avoid dot-dot (or path-traversal) attacks. Your file name should never have dot-dot (i.e. "..") in it.

File not found (404)


To output File Not Found message (i.e. 404 code back to the web client), use:
cld_cant_find_file ("Explain reason for Not Found");
To use this function, HTTP header must not have been output.

Flush output


The program output can be flushed with  cld_flush_printf:
int num_bytes_written = cld_flush_printf(0);
The input parameter to this function is always 0. Returns -1 if no output could be flushed (meaning HTML output is disabled and no writing to string is happening now), or number of bytes written (0 or more).
Typically you don't need to call this function, and it may slow down the output if you do, even if your output is large, since flushing is done automatically when needed. In rare occassions you might use it, such as if your output consists of a number of parts and each part takes some time to compute.

Create new database document and a new associated file


Use cld_make_document:
define-string document_id

// do the rest in a C block
start-c

char document_file[MAX_FILENAME_LEN + 1];

// Create a new document. You get:
// 1. document_id - which is a unique ID for the document.
// 2. document_file - a full-path file name associated with this unique ID for the document
// You likely want to store both document_id and document_file in some table where you can reference them later.

FILE *f = cld_make_document (&document_id, document_file, MAX_FILENAME_LEN);

// Now you can  write to the file with file pointer f, close it, etc.
fprintf (f, "Hello!");
fclose(f);
end-c

Strings

Initialize, Copy, Append strings


The following functions are used for string creating and manipulation:


Output data to string


Program output normally goes to the web client. You can redirect it to a string by using:
char *my_str;

cld_write_to_string (&my_str);

cld_printf (CLD_NOENC, "Some output that normally goes out to the client..");

cld_write_to_string (NULL);

int bytes_written = cld_write_to_string_length ();
In the above example, the first call to cld_write_to_string signifies that all output will go to variable str. This is so until the next call to cld_write_to_string happens with NULL parameter. The length of the string written can be obtained immediately after the closing call with cld_write_to_string_length.

Calls to cld_write_to_string can be nested, i.e.:
char *my_str;

cld_write_to_string (&my_str);

cld_printf (CLD_NOENC, "This output goes to my_str variable..");

char *my_str_1;

cld_write_to_string (&my_str_1);
cld_printf (CLD_NOENC, "This output goes to my_str_1 variable...");
cld_write_to_string (NULL);
int bytes_written_1 = cld_write_to_string_length ();

cld_write_to_string (NULL);

int bytes_written = cld_write_to_string_length ();
In this case the second cld_printf goes to my_str_1 variable, and not to my_str variable. Variable bytes_written_1 has the length of this output only. The result for the first cld_printf is the same as before.

cld_write_to_string can be used to capture page output:
define-string my_str

c cld_write_to_string (&my_str);

Hello!<br/>
<br/>
It is a good day today!<br/>
<hr/>

c cld_write_to_string (NULL);

print-noenc my_str
In this above example, the output (Hello!... up to and including <hr/>) is written into my_str string, which is then output.

Change case of strings (upper, lower)


To lower or uppercase a string, use:
char my_str[] = "SOME STRING";
cld_lower(my_str);
Variable my_str is now "some string".
cld_upper(my_str);
Variable my_str is now "SOME STRING".

Both cld_lower and cld_upper return the input string, in this case they would both return my_str.

Substrings



Trim string


To trim string of whitespace both on left and right, use cld_trim:
char *str = cld_strdup (" ABC ");
int len = strlen (str);

cld_trim (str, &len);
Variable str will be "ABC" and len will be 3 (i.e. the new length). This function doesn't return a value.  

Break down string into an array of strings based on a delimiter


Often times a string needs to be broken down into pieces based on a variable delimiter. To do that, use cld_break_down:
define-string to_break="name=value+name1=value1+name2=value2"
start-c
  cld_broken broken;
  cld_break_down (to_break, "+", &broken);
  int i;

  for (i = 0; i < broken.num_pieces; i++)
  {
       cld_broken broken_further;
       cld_break_down (broken.pieces[i], "=", &broken_further);
       cld_printf (CLD_WEB, "Name is %s, value is %s", broken_further.pieces[0], broken_further.pieces[1]);
       cld_printf (CLD_NOENC, "<br/>\n");
  }
end-c
The output would be:

Name is name, value is value
Name is name1, value is value1
Name is name2, value is value2

The string to_break is broken by inserting zero characters, for better performance. If you want it unchanged, use cld_strdup to make a copy first, and then break down the copy. The variable 'broken' (of cld_broken type) has num_pieces member (the number of pieces broken down to) and pieces (the array of pieces).  

In the above example, we first break down the original string into pieces separated by "+", and then break those down separated by "=" (in general the separator is any string). Since the original string is broken down, you can also traverse the pieces by reading them zero-terminated one by one from the original string to_break.

This is an efficient way of breaking down a string without making any copies.

Clear Unused variable error


It is a good practice to delete unused variables, but sometimes they cannot be. To prevent compiler warnings, use:
CLD_UNUSED(x);

Error reporting


Fatal errors (resulting in program termination) can be reported by cld_report_error which takes the arguments the same as printf and returns number of bytes written:
cld_report_error("Error %s happened, terminating", err);
To report errors and do not exit, use:
cld_report_error_no_exit("Error %s happened, continuing", err)
In either case, any currently open database transactions are rolled back.

Encryption and hashing

Cloudgizer uses 256 bit SHA hashing and AES encryption. The cipher used is AES-256-CBC.

Hashing


To create a 256-bit SHA hash, use cld_sha:
char *hash = cld_sha("some value to hash");
Variable hash now contains hashed value of "some value to hash". This is a string terminated by null character, always 64 bytes in length. Hashed value might look something like 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'.

Encrypting data


Use cld_aes_encrypt, for example:
char *to_encrypt = "some data to encrypt";
int len = strlen(to_encrypt);
EVP_CIPHER_CTX e_ctx;
cld_get_enc_key("password", "salt", &e_ctx, NULL);
char *encrypted_data = cld_aes_encrypt(&e_ctx, (unsigned char*)to_encrypt, &len, 0);
The output of the function ('encrypted_data') contains encrypted data, which is allocated by Cloudgizer.

In  the above example, we first create an encryption context out of password and salt ("password" and "salt" in the above example) and encrypt string 'to_encrypt' of length 'len'. The fourth parameter is 0 (character mode of encryption), meaning the result is the character string (i.e. the encrypted value is null-terminated and suitable for using in database varchar columns, for example).  If the fourth parameter is 1 (binary mode of encryption), the result is a binary encrypted string.

If it is character mode, then the encrypted output will be converted into a string of hexadecimal characters (i.e. ranging from '0' to '9' and from 'a' to 'f') terminated by null character. The maximum size of a buffer of encrypted string is 2*(length_of_input_string+16)+1. This accounts for the maximum of 16 bytes extra for AES, and double for hexadecimal representation (in case of character mode of encryption).  Character mode of encryption is convenient if the result of encryption should be a human readable string, or for the purposes of non-binary storage in the database.

Variable 'len' holds the length (in bytes) of data to encrypt, and on return it holds the length (in bytes) of the encrypted data (excluding zero byte at the end if the fourth parameter is 0, i.e. if the result is a character string).

The salt can be NULL in which case no salt is used.

Decrypting data


Use cld_aes_decrypt. If 'encrypted_data' is encrypted data produced by cld_aes_encrypt:
EVP_CIPHER_CTX d_ctx;
cld_get_enc_key("password", "salt", NULL, &d_ctx);
int len = strlen (encrypted_data);
char *decrypted_data = cld_aes_decrypt(&d_ctx, (unsigned char*)encrypted_data, &len, 0);
The output of the function ('decrypted_data') contains decrypted data, which is allocated by Cloudgizer.

In  the above example, we first create a decryption context out of password and salt ("password" and "salt" in the above example) and decrypt string 'encrypted_data' of length 'len'. The fourth parameter is 0, meaning that 'encrypted_data' is a character string produced by cld_aes_encrypt with fourth parameter of 0. When decrypting, password, salt and the mode of encryption (binary or character) must match between cld_aes_encrypt and cld_aes_decrypt.

Variable 'len' holds the length (in bytes) of encrypted data (excluding zero byte at the end in case of character mode of encryption), and on return it holds the length (in bytes) of the decrypted data (excluding zero byte at the end which is always placed at the end of decrypted data).

Generating random string


To generate a random string (null-terminated) use cld_make_random(). For example, to generate a random string, use:
char rnd[20];
cld_make_random (rnd, sizeof(rnd));
Variable 'rnd' now holds random string of length 19. The buffer 'rnd' must be at least a byte longer than the length of random string you desire, to accomodate null terminator.

Storing and retrieving data in a sequential list (FIFO)


To store data in a sequential list, and be able to rewind to the beginning and retrieve later, use functions related to cld_store_data type:
// Declare list data
cld_store_data list_data;

// Initialize list data
cld_store_init (&list_data);

// Store name/value pairs to list data
cld_store (&list_data, "item name 1", "item value 1");
cld_store (&list_data, "item name 2", "item value 2");
cld_store (&list_data, "item name 3", "item value 3");
...

// Rewind list data to the beginning
cld_rewind (&list_data);

// Retrieve name/value pairs in the order in which they were put in
char *item_name;
char *item_value;
while (1)
{
   cld_retrieve (&list_data, &item_name, &item_value);
   if (name == NULL) break;
   //
   // item_name and item_value will be "item name 1"/"item value 1", "item name 2"/"item value 2" etc.
   //
}

// Delete list data
cld_purge (&list_data);

Executing Cloudgizer program from command line (Batch processing)

Using batch processing


Batch processing is using the program as a command line program. To enable batch processing, use cld_enable_batch_processing:
cld_enable_batch_processing();
Batch processing disables web output (HTML output). Only output to a string will work, while web output will not. You can output to stdout and stderr (either via C functions like printf or via markups like print-out and print-error), and read from stdin. A batch program can still receive input parameters via URL, through environment variables, much like web server would:
export REQUEST_METHOD=GET
export QUERY_STRING="page=home&action=employees"
export CLD_BATCH_PROCESSING=yes
./your_program
When writing QUERY_STRING, make sure that actual data is URL encoded. For that, use -urlencode option for cld command line utility, like in this example:
export REQUEST_METHOD=GET
export HELLO_WORLD_URL_ENCODED=$(cld -urlencode "hello world")
export QUERY_STRING="page=key&dt=${HELLO_WORLD_URL_ENCODED}&action=file"
export CLD_BATCH_PROCESSING=yes
./your_program
cld -urlencode "hello world" will output URL encoded "hello world" string, with space substituted with '%20'. This output is stored into an environment variable, and its contents then used in QUERY_STRING as URL-encoded string for a URL passed to your program.

In the above bash example, REQUEST_METHOD is set to GET, and QUERY_STRING is set to request's input parameters in URL notation. You must set CLD_BATCH_PROCESSING to yes, and examine it in your program:
...
const char *batch_env = cld_ctx_getenv ("CLD_BATCH_PROCESSING");
if (!strcasecmp (batch_env, "yes"))
{
   cld_enable_batch_processing();
   /*<
   input-param action
   
   if-string action="employees"
       write-string define employees
           List of employees
           <hr/>
           run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
               query-result#my_query, name as define name
               query-result#my_query, yearOfHire as define year_of_hire
               print-web name
               ,
               print-web year_of_hire
               <br/>
           end-query
           <hr/>
       end-write-string
       print-out "%s", employees
   end-if
   >*/
   return;
}
...
In this program, cld_ctx_getenv is first used to enable batch processing with cld_enable_batch_processing, meaning there is no web output. However, you can use most all other features such as working with input parameters, writing strings, queries etc. In the end, the program outputs to standard output:

List of employees
<hr/>
Linda,2001<br/>
John,2003<br/>
<hr/>

Batch programs can be often written together with web programs if their business logic is intertwined. Effectively you can write programs that could be used as both web and command-line programs, for example:
...
const char *batch_env = cld_ctx_getenv ("CLD_BATCH_PROCESSING");
int is_batch = !strcasecmp (batch_env, "yes"));

if (is_batch)
{
   cld_enable_batch_processing();
}
else
{
   /*< output-http-header >*/
}

/*<
input-param action

if-string action="employees"
   write-string define employees
       List of employees
       <hr/>
       run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
           query-result#my_query, name as define name
           query-result#my_query, yearOfHire as define year_of_hire
           print-web name
           ,
           print-web year_of_hire
           <br/>
       end-query
       <hr/>
   end-write-string
   if is_batch
       print-out " %s\n", employees
   else
       print-noenc employees
   end-if
end-if
>*/
...
The above program can be called both from the web client to output a list of employees, and it can output this list to standard output. The URL parameters used are the same. Batch program's output would be the same as above, while the output to web client would be (for URL such as http://myserver.com/go.your_app_name?page=home&action=employees):

List of employees
Linda,2001
John,2003

Exit code for batch processing


To set exit code for a command-line program, use:
cld_set_exit_code(3);
This sets exit code to 3.

POST URL (web call)


To perform a web call via HTTP/HTTPS:
char *response;
char *error;
char *cert;
int result = cld_post_url_with_response ("https://somesite.com?par=3", &response, &error, &cert);
The function returns 1 on success and 0 on failure. Variable error has error text in case of failure. If HTTPS is used the location of certificate file can be provided in variable cert. If cert is NULL, certificate is not checked. The actual response (minus headers) is stored in variable response. Response that redirects to another URL will be followed up to 4 times. The response is the body message, not the headers.

Files

File storage


You can read and write any files accessible to you, using either standard C mechanisms, or the file API explained in this documentation.

You can also use a provided file storage under 'file' directory - see more about this file storage.

To create a file in this file storage, use the following code:
define-string document_id
c char document_file[MAX_FILENAME_SIZE];
c FILE *f = cld_make_document (&document_id, document_file, MAX_FILENAME_SIZE);
Output variables are document_id (containing the unique ID of the file created, produced by using cldDocumentGenerator table), document_file (containing unique file name with a complete path to it), and file descriptor 'f' as the file created is opened and ready to write to.

You'd typically store file ID and its path in the database, and use them to later manipulate the file (read, write, delete etc.).

Get file size


To get the size of a file:
size_t sz = cld_get_file_size ("/home/user/my_file");
The return value will be -1 if cannot open file or its size.

Copy files

int result = cld_copy_file ("/home/user/from_file", "/home/user/to_file");
This copies from file from_file to file to_file. The result is -1 if cannot open from_file, -2 if cannot open to_file, -3 if cannot read from_file, -4 if cannot write to_file, and 1 on success.

Read entire file into a string variable

char *file_data;
int result = cld_read_whole_file ("/home/path/my_file", &file_data);
This reads file my_file into string variable file_data, which is cld_malloc() allocated (like all Cloudgizer memory is). Variable 'result' is -1 if cannot open my_file, -2 if cannot read my_file, and the size of the file read (0, or a positive number) if reading succeeded.  

Write whole file from a string

char content[] = "some content";
size_t content_len = strlen(content);

int result = cld_write_file ("/home/path/my_file", content, content_len, 0);
This writes "some content" to my_file. Variable 'result' is -1 if cannot open my_file, -2 if cannot write it, and 1 if writing succeeded. The whole file is overwritten with the new content.

Append to file from a string

char content[] = "new content";
size_t content_len = strlen(content);

int result = cld_write_file ("/home/path/my_file", content, content_len, 1);
This appends "new content" to the end of my_file. Variable 'result' is -1 if cannot open my_file, -2 if cannot write it, and 1 if writing succeeded.

Lock a file


You can create a file and lock it so your process has exclusive read and write lock on it. This can be useful as a process control means, for example to ensure only one process can continue work.
// returns 0 if cannot lock, -1 if cannot open file, 1 if locked,-2 invalid path
int file_descriptor;
int result = cld_lockfile ("/home/user/file_to_create", &file_descriptor);
The function returns -2 if file path is invalid, -1 if cannot open file, 0 if cannot lock, and 1 if locked.

The process must not close stdout, stderr or stdin while file lock is in effect.

The file stays locked until the program ends, or until it's closed, for instance:
close (file_descriptor);

Get application home directory


Application's home directory is a directory named after application name location directly under the home directory of the user who executes the program:
char *home_directory = cld_home_dir ();
If cannot determine application home directory, empty string ("") is returned.

Application's home directory is based on application name. For example, if variable cld_handler_name is 'your_app_name' (which is the same as CLD_APP_NAME in appinfo during installation), then application home directory is /home/user/your_app_name. Do not ever change variable cld_handler_name.

Check if directory


To determine if something is a directory:
int is_dir = cld_is_directory ("/home/user/something");
Variable is_dir will be 1 if something is a directory, 0 otherwise.  

Temporary files

To store temporary files, use temporary directory:
print-web cld_get_config()->app->tmp_directory

Send email


To send email via sendmail, use:
const char *from = "addr1@something.com"
const char *to = "addr2@somethingelse.com"
int result = cld_sendmail (from, to, "subject of message", NULL, "This is the body of email message");
Variable 'result' is the exit code of sendmail program. In the previous example, 4th parameter was NULL, meaning no additional header was added to the email. Email sender is 'from' and recipient 'to', followed by the subject of the message, while the message itself is the 5th parameter.

If you want to add custom headers to the email sent (for example to send HTML email, for attachments etc.), you can add them:
const char *from = "addr1@something.com"
const char *to = "addr2@somethingelse.com"
cons char *headers= "MIME-Version: 1.0\r\nContent-Type: text/html";
int result = cld_sendmail (from, to, "subject of message", headers, "This is the body of email message");

Queries and Transactions

SELECT from the database

int nrow;
int ncol;
char **col_names;
char **data;

// To select data
cld_select_table ("SELECT ...", &nrow, &ncol, &colnames, &data);
'nrow' is the number of rows returned. 'ncol' is the number of columns. 'colnames' is a list of column names returned. 'data' holds all column data in order in which it is retrieved (first all columns from the first row, then from the second etc.), for example if there are two columns returned then data[0] is the first column of the first row, data[3] is the second column of the second row etc.

'data' can be NULL, while 'colnames' cannot be NULL. Maximum  number of output columns is 1000.

To get column names only, without the data:
cld_select_table ("SELECT ...", &nrow, &ncol, &col_names, NULL);
In this case, col_names[0] is the name of the first column, col_names[1] is the name of the second column and so forth.

This function does not return value, and its failure will stop the program (i.e. it is expected to always succeed).  

Get value from auto-increment column


Insert into a table with auto increment column produces new value for it. To obtain it:
char val[100];
cld_get_insert_id (val, sizeof(val));
'val' is the output buffer. This function must be called immediately after INSERT and it does not return a value.

Execute any SQL other than SELECT, including DDL statements

int rows;
unsigned int er;
char *err_message;
if cld_execute_SQL ("INSERT INTO ... ",  &rows, &er, &err_message)==1
   print-web "Success"
else
   print-web "Failure"
end-if
Varable 'rows' is the number of rows affected, 'er' is the error number, and 'err_message' is the error message. If failed, the function returns 0, otherwise 1.

Use this to execute any SQL, including DDL (Data Definition Language) such as CREATE TABLE, DROP TABLE etc.

Check for open transaction

if cld_check_transaction(1)==1
   print-web "You have an open transaction!"
end-if
If input parameter is 1, the function returns 1 if there is an open transaction and 0 if not. If input parameter is 0 then it will report an error if there is an open transaction and it will return 0 if not.

If input parameter is 2, and there is an open transaction, rollback this transaction and return 0.

Get miscellaneous

Get web address for forms and links


Web address (for forms and links) is obtained from config file under web_address field. You can get it from:
const char *web_address = cld_web_address ();

Get Base URL of server


Base URL for "https://myserver.com/go.your_app_name?par1=val1" is "myserver.com". To obtain it, use cld_web_name and pass any URL leading to the server:
char *base_URL = cld_web_name(cld_web_address());
This will produce the base name of a web server running the program.

Get version


The software version can be obtained via following call:
const char *major_version = cld_major_version();

Get current, past or future time, including GMT

To get current, past or future time, suitable for cookies, use cld_time:
char *cld_time (const char *timezone, int year, int month, int day, int hour, int min, int sec);
Returns time (now, in the future, or the past). Input parameter 'timezone' is the name of TZ (timezone). So to get GMT time then timezone should be "GMT", if it is Mountain Standard then it is "MST" etc.

Input parameters year,month,day,hour,min,sec are the time to add to current time (they can be negative too). So for example ..(0,0,1,0,0,1) adds 1 day and 1 second to the current time, while ..(0,-1,0,0,0) is one month in the past.

Get current time


To get current time, per local time zone, use cld_current_time. The arguments are the buffer where the time should be stored and its length:
char buf[50];
cld_current_time (buf, sizeof(buf));
The time returned shows year, month, day of month, hour, minute, second, separated by a dash, such as '2017-07-30-18:48:13', with hours in 0-24 range.

Get application name


Application name is stored in global variable cld_handler_name, this is the same as CLD_APP_NAME in appinfo file during installation.
/*<
 Application name is <?print-web cld_handler_name?>
>*/
Application's home directory is based on it. For example, if cld_handler_name is 'your_app_name', then application home directory is /home/user/your_app_name.

Never change cld_handler_name.

Get environment variable (web or command shell)


Environment variables can be available from the operating system or the web server. If the program is running in batch mode (i.e. command line), the environment used is that of the OS. When a program is running within a web server, first web server's variables are searched for a match, and if not found, the OS environment is used. The call that does this is cld_ctx_getenv:
const char *env_value = cld_ctx_getenv ("CONTENT_TYPE");

Installing from source code


For installing Cloudgizer and the Example Application, it is recommended you follow Cloudgizer download and installation and Example Application download and installation.

If you want to install them  from source code, use the following instructions.

You need a server or a VPS (virtual private server) with RH/Centos 7 installed and Internet connectivity.

First you might need to install wget and git if they are not already:
sudo yum -y install git
sudo yum -y install wget

Cloudgizer

To make the installation file from source code:

  • Setup tools, libraries and the database:
    sudo ./setup_env
    sudo ./setup_maria
    sudo ./setup_postfix
  • Create Cloudgizer installation file:
    ./pack_all
    This will install Cloudgizer and also create cloudgizer.tar.gz installation file you can use to install Cloudgizer elsewhere, see for details on that - the instructions are the same except that you already have cloudgizer.tar.gz and you won't use wget to get it.

  • Example Application

    Follow these instructions to make the installation file from source code. You must have Cloudgizer installed first.


  • Create installation file:
    cldpackapp

  • Now you have example.tar.gz file that you can use to install Example Application, see for details on that - the instructions are the same except that you already have example.tar.gz and you won't use wget to get it.


    Copyright (c) Zigguro LLC 2017. Cloudgizer is a trademark of Zigguro LLC. Contact email address admin@zigguro.org. The Zigguro software and information herein are provided "AS IS" and without any warranties or guarantees of any kind.