/ 
Config-JSON-Enhanced-0.10
River stage one • 1 direct dependent • 1 total dependent
⭐ Starred 0 GitHub stars
/ README.md

NAME

Config::JSON::Enhanced - JSON-based config with C/Shell-style comments, verbatim sections and variable substitutions

VERSION

Version 0.10

SYNOPSIS

This module provides subroutine config2perl() for parsing configuration content, from files or strings, based on, what I call, "enhanced JSON" (see section "ENHANCED JSON FORMAT" for more details). Briefly, it is standard JSON which allows:

This module was created because I needed to include long shell scripts containing lots of quotes and newlines, in a configuration file which started as JSON.

The process is simple: so-called "enhanced JSON" is parsed by config2perl. Comments are removed, variables are substituted, verbatim sections become one line again and standard JSON is created. This is parsed with JSON (via Data::Roundtrip::json2perl) to produce a Perl data structure which is returned.

It has been tested with unicode data (see t/070-config2perl-complex-utf8.t) with success. But who knows ?!?!

Here is an example:

   use Config::JSON::Enhanced;

   # simple "enhanced" JSON with comments in 3 styles: C,shell,CPP
   my $configdata = <<'EOJ';
    {
       /* 'a' is ... */
       "a" : "abc",
       # b is ...
       "b" : [1,2,3],
       "c" : 12 // c is ...
    }
   EOJ
   my $perldata = config2perl({
       'string' => $configdata,
       'commentstyle' => "C,shell,CPP",
   });
   die "call to config2perl() has failed" unless defined $perldata;
   # the standard JSON:
   # {"a" : "abc","b" : [1,2,3], "c" : 12}


   # this "enhanced" JSON demonstrates the use of variables
   # which will be substituted during the transformation to
   # standard JSON with user-specified data.
   # Notice that the opening and closing tags enclosing variable
   # names can be customised using the 'tags' input parameter,
   # so as to avoid clashes with content in the JSON.
   my $configdata = <<'EOJ';
    {
      "d" : [1,2,<% tempvar0 %>],
      "configfile" : "<%SCRIPTDIR%>/config/myapp.conf",
      "username" : "<% username %>"
       }
    }
   EOJ
   my $perldata = config2perl({
       'string' => $configdata,
       'commentstyle' => "C,shell,CPP",
       # optionally customise the tags enclosing the variables
       # when you want to avoid clashes with other strings in JSON
       #'tags' => ['<%', '%>'], # <<< these are the default values
       # user-specified data to replace the variables in
       # the "enhanced" JSON above:
       'variable-substitutions' => {
           'tempvar0' => 42,
           'username' => getlogin(),
           'SCRIPTDIR' => $FindBin::Bin,
       },
   });
   die "call to config2perl() has failed" unless defined $perldata;
   # the standard JSON
   # (notice how all variables in <%...%> are now replaced):
   # {"d" : [1,2,42],
   #  "username" : "yossarian",
   #  "configfile" : "/home/yossarian/B52/config/myapp.conf"
   # }


   # this "enhanced" JSON demonstrates "verbatim sections"
   # the puprose of which is to make more readable JSON strings
   # by allowing them to span over multiple lines.
   # There is also no need for escaping double quotes.
   # template variables (like above) will be substituted
   # There will be no comments removal from the verbatim sections.
   my $configdata = <<'EOJ';
    {
     "a" : <%begin-verbatim-section%>
     This is a multiline
     string
     "quoted text" and 'quoted like this also'
     will be retained in the string escaped.
     White space from beginning and end will be chomped.

     <%end-verbatim-section%>
     ,
     "b" = 123
    }
   EOJ
   my $perldata = config2perl({
       'string' => $configdata,
       'commentstyle' => "C,shell,CPP",
   });
   die "call to config2perl() has failed" unless defined $perldata;
   # the standard JSON (notice that "a" value is in a single line,
   # here printed broken for readability):
   # {"a" :
   #   "This is a multiline\nstring\n\"quoted text\" and 'quoted like
   #   this also'\nwill be retained in the string escaped.\nComments
   #   will not be removed.\nWhite space from
   #   beginning and end will be chomped.",
   #  "b" : 123
   # };

EXPORT

SUBROUTINES

config2perl

my $ret = config2perl($params);
die unless defined $ret;

Arguments:

Return value:

Given input content in "ENHANCED JSON FORMAT", this sub removes comments (as per preferences via input parameters), replaces all template variables, if any, compacts "Verbatim Sections", if any, into a single-line string and then parses what remains as standard JSON into a Perl data structure which is returned to caller. JSON parsing is done with Data::Roundtrip::json2perl, which uses JSON.

Comments outside of JSON fields will always be removed, otherwise JSON can not be parsed.

Comments inside of JSON fields, keys, values, strings etc. will not be removed unless input parameter remove-comments-in-strings is set to 1 by the caller.

Comments (or what looks like comments with the current input parameters) inside "Verbatim Sections" will never be removed.

The input content to-be-parsed can be specified with one of the following input parameters (entries in the $params):

Additionally, input parameters can contain the following keys:

See section "ENHANCED JSON FORMAT" for details on the format of what I call enhanced JSON.

config2perl returns the parsed content as a Perl data structure on success or undef on failure.

ENHANCED JSON FORMAT

This is JSON with added reasonable, yet completely ad-hoc, enhancements (from my point of view).

These enhancements are:

Verbatim Sections

A Verbaitm Section in this ad-hoc, so-called Enhanced JSON is content enclosed within <%begin-verbatim-section%> and <%end-verbatim-section%> tags. A verbatim section's content may span multiple lines (which when converted to JSON will preserve them by escaping. e.g. by replacing them with '\n') and can contain template variables to be substituted with user-specified data. All single and double quotes can be left un-escaped, the program will escape them (hopefully correctly!).

The content of Verbatim Sections will have all its template variables substituted. Comments will be left untouched.

The tags for denoting the opening and closing a verbatim section are controled by the 'tags' parameter to the sub config2perl. Defaults are <% and %>.

Here is an example of enhanced JSON which contains comments, a verbatim section and template variables:

my $con = <<'EOC';
{
  "long bash script" : ["/usr/bin/bash",
/* This is a verbatim section */
<%begin-verbatim-section%>
  # save current dir, this comment remains
  pushd . &> /dev/null
  # following quotes will be escaped
  echo "My 'appdir' is \"<%appdir%>\""
  echo "My current dir: " $(echo $PWD) " and bye"
  # go back to initial dir
  popd &> /dev/null
<%end-verbatim-section%>
/* the end of the verbatim section */
  ],
  // this is an example of a template variable
  "expected result" : "<% expected-res123 %>"
}
EOC

# Which, can be processed thusly:
my $res = config2perl({
  'string' => $con,
  'commentstyle' => 'C,CPP',
  'variable-substitutions' => {
    'appdir' => Cwd::abs_path($FindBin::Bin),
    'expected-res123' => 42
  },
});
die "call to config2perl() has failed" unless defined $res;

# following is the dump of $res, note the breaking of the lines
# in the 'long bash script' is just for readability.
# In reality, it is one long line:
{
  "expected result" => 42,
  "long bash script" => [
    "/usr/bin/bash",
    "# save current dir, this comment remains\npushd . &> /dev/null\n
     # following quotes will be escaped\necho \"My 'appdir' is
     \\\"/home/babushka/Config-JSON-Enhanced/t\\\"\"\n
     echo \"My current dir: \" \$(echo \$PWD) \" and bye\"\n# go back to
     initial dir, this comment remains\npopd &> /dev/null"
  ]
};

A JSON string can contain comments which you may want to retain (note: comments filtering will not apply to verbatim sections).

For example if the content is a unix shell script it is possible to contain comments like # comment. These will be removed along with all other comments in the entire JSON input if you are using shell style comments. Another problem is when JSON string contains comment opening or closing tags. For example consider this cron spec : */15 * * * * which contains the closing string of a C-style comment and will cass a big mess.

You have two options in order to deal with this problem:

TIPS

You can change the tags used in denoting the template variables and verbatim sections with the tags parameter to the sub config2perl. Use this feature to change tags to something else if your JSON contains the same character sequence for these tags and avoid clashes and unexpected substitutions. <% and %> are the default tags.

Similarly, custom comment style (specifying what should be the opening and, optionally, closing tags) can be employed if your JSON strings contain something that looks like comments and you want to avoid their removal.

WARNINGS/CAVEATS

In order to remove comments within strings, a simplistic regular expression for extracting quoted strings is employed. It finds anything within two double quotes. It tries to handle escaped quotes within quoted strings. This regex may be buggy or may not be complex enough to handle all corner cases. Therefore, it is possible that setting parameter remove-comments-in-strings to 1 to sub config2perl to cause unexpected results. Please report these cases, see SUPPORT.

The regex for identifying comments, variables and verbatim sections has the custom tags escaped for special regex characters (with the \Q ... \E construct). So you are pretty safe in using any character. Please report weird behaviour.

AUTHOR

Andreas Hadjiprocopis, <bliako at cpan.org>

HUGS

! Almaz !

BUGS

Please report any bugs or feature requests to bug-config-json-enhanced at rt.cpan.org, or through the web interface at https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Config-JSON-Enhanced. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

perldoc Config::JSON::Enhanced

You can also look for information at:

ACKNOWLEDGEMENTS

LICENSE AND COPYRIGHT

This software is Copyright (c) 2023 by Andreas Hadjiprocopis.

This is free software, licensed under:

The Artistic License 2.0 (GPL Compatible)