NAME
Data::Secs2 - canoncial string for nested data
SYNOPSIS
#####
# Subroutine interface
#
use Data::Secs2 qw();
@array = arrayify( @var );
$string = itemify($format, @elements);
$string = itemify($format, @elements, [@options]);
$string = itemify($format, @elements, {optioins});
\@sec_list = listify(@vars);
\@sec_list = neuterify($binary_secs);
\@sec_list = neuterify($binary_secs, @options);
\@sec_list = neuterify($binary_secs, [@options]);
\@sec_list = neuterify($binary_secs, {@options});
$error = scalarize( \@sec_list );
$ascii_secs = secsify( \@sec_list);
$ascii_secs = secsify( \@sec_list, @options);
$ascii_secs = secsify( \@sec_list, [@options]);
$ascii_secs = secsify( \@sec_list, {@options});
$binary_secs = secsify( \@sec_list, type => 'binary');
$binary_secs = secsify( \@sec_list, type => 'binary', @options);
$binary_secs = secsify( \@sec_list, [type => 'binary',@options]);
$binary_secs = secsify( \@sec_list, {type => 'binary',@options});
$string = stringify( @arg );
\@sec_list = transify($acsii_secs);
\@sec_list = transify($acsii_secs, @options);
\@sec_list = transify($acsii_secs, [@options]);
\@sec_list = transify($acsii_secs, {@options});
$error = vectorize( \@sec_list );
#####
# Class interface
#
use Data::Secs2;
@array = Data::Secs2->arrayify( @var );
$string = Data::Secs2->itemify($format, @elements);
$string = Data::Secs2->itemify($format, @elements, [@options]);
$string = Data::Secs2->itemify($format, @elements, {optioins});
\@sec_list = Data::Secs2->listify(@vars);
\@sec_list = Data::Secs2->neuterify($binary_secs);
\@sec_list = Data::Secs2->neuterify($binary_secs, @options);
\@sec_list = Data::Secs2->neuterify($binary_secs, [@options]);
\@sec_list = Data::Secs2->neuterify($binary_secs, {@options});
$error = Data::Secs2->scalarize( \@sec_list );
$ascii_secs = Data::Secs2->secsify( \@sec_list);
$ascii_secs = Data::Secs2->secsify( \@sec_list, @options);
$ascii_secs = Data::Secs2->secsify( \@sec_list, [@options]);
$ascii_secs = Data::Secs2->secsify( \@sec_list, {@options});
$binary_secs = Data::Secs2->secsify( \@sec_list, type => 'binary');
$binary_secs = Data::Secs2->secsify( \@sec_list, type => 'binary', @options);
$binary_secs = Data::Secs2->secsify( \@sec_list, [type => 'binary',@options]);
$binary_secs = Data::Secs2->secsify( \@sec_list, {type => 'binary',@options});
$string = Data::Secs2->stringify( @arg );
\@sec_list = Data::Secs2->transify($acsii_secs);
\@sec_list = Data::Secs2->transify($acsii_secs, @options);
\@sec_list = Data::Secs2->transify($acsii_secs, [@options]);
\@sec_list = Data::Secs2->transify($acsii_secs, {@options});
$error = Data::Secs2->vectorize( \@sec_list );
DESCRIPTION
The 'Data::SECS2' module provides a widely accepted method of packing nested lists into a linear string and unpacked nested lists. In the hardware world, data and data passed between hardware is not stored in SQL style tables but nested lists. Nested data has a long history in mathematics.
The Data::Secs2 program module facilitates the secsification of the nested data is in accordance with SEMI E5-94, Semiconductor Equipment Communications Standard 2 (SECS-II), pronounced 'sex two' with gussto and a perverted smile. The SEMI E4 SECS-I standard addresses transmitting SECSII messages from one machine to another machine (all most always host to equipment) serially via RS-232. And, there is another SECS standard for TCP/IP, the SEMI E37 standard, High-Speed SECS Message Services (HSMS) Generic Services.
In order not to plagarize college students, credit must be given where credit is due. Tony Blair, when he was a college intern at Intel Fab 4, in London invented the SEMI SECS standards. When the Intel Fab 4 management discovered Tony's secsification of their host and equipment, they called a board of directors meeting, voted, and elected to have security to escort Tony out the door. This was Mr. Blair's introduction to voting and elections which he leverage into being elected prime minister of all of England. In this new position he used the skills he learned at the Intel fab to secsify intelligence reports on Iraq's weopons of mass distruction.
By using a well-known, widely-used standard invented and publicized by the prime mister of England for packing and unpacking Perl nested data, not only is this useful in a open nested operations such as comparing nested data, storing the packed data in a file, but also for transmitting data from one Perl site to another or between Perl and other programming languages.
And do not forget the added benefit of SEMI secs humor and that the real originators of the SECS-II yielded and allowed Tony Blair to take illegal credit for SECS-II.
SECSII Format
The nested data linear format used by the Data::Secs2 suroutines is in accordance with SEMI E5-94, Semiconductor Equipment Communications Standard 2 (SECS-II), pronounced 'sex two' with gussto and a perverted smile. This industry standard is copyrighted and cannot be reproduced without violating the copyright. However for those who have brought the original hard media copy, there are robot help and Perl POD open source copyrighted versions of the SECII hard copy copyright version available. The base copyright is hard copy paper and PDF files available from
Semiconductor Equipment and Materials International
805 East Middlefield Road,
Mountain View, CA 94043-4080 USA
(415) 964-5111
Easylink: 62819945
http://www.semiconductor-intl.org
http://www.reed-electronics.com/semiconductor/
The SEMI E4 SECS-I standard addresses transmitting SECSII messages from one machine to another machine (all most always host to equipment) serially via RS-232. And, there is another SECS standard for TCP/IP, the SEMI E37 standard, High-Speed SECS Message Services (HSMS) Generic Services.
In order not to plagarize college students, credit must be given where credit is due. Tony Blair, when he was a college intern at Intel Fab 4, in London invented the SEMI SECS standards. When the Intel Fab 4 management discovered Tony's secsification of their host and equipment, they elected to have security to escort Tony out the door. This was Mr. Blair's introduction to elections which he leverage into being elected prime minister. In this new position he used the skills he learned at the Intel fab to secsify intelligence reports on Iraq's weopons of mass distruction.
The SEMI E5 SECS-II standard is a method of forming listified packed messages from nested list data. It consists of elements where each element is a format code, number of elements followed by the elements.
Table 1 Item Format Codes
unpacked binary octal hex description
----------------------------------------
L 000000 00 0x00 List (length in elements)
B 001000 10 0x20 Binary
T 001001 11 0x24 Boolean
A 010000 20 0x40 ASCII
J 010001 21 0x44 JIS-8
S8 011000 30 0x60 8-byte integer (signed)
S1 011001 31 0x62 1-byte integer (signed)
S2 011010 32 0x64 2-byte integer (signed)
S4 011100 34 0x70 4-byte integer (signed)
F4 100000 40 0x80 8-byte floating
F8 100100 44 0x90 4-byte floating
U8 101000 50 0xA0 8-byte integer (unsigned)
U1 101001 51 0xA4 1-byte integer (unsigned)
U2 101010 52 0xA8 2-byte integer (unsigned)
U4 101100 54 0xB0 4-byte integer (unsigned)
Notes:
ASCII format - Non-printing characters are equipment specific
Integer formats - most significant byte sent first
floating formats - IEEE 753 with the byte containing the sign sent first.
Nested lists are packed is a linear number of items where each item consists of a header followed by the elements in the item. The header is of the following format where the MS byte is always first when transmitting the bytes linearally:
bits MSB LSB
7 6 5 4 3 2 1 0
+-------+-------+-------+-------+-------+-------+-------+-------+
| Format code |# length bytes |
+---------------------------------------------------------------+
|MSB MS length byte LSB |
+---------------------------------------------------------------+
| length byte |
+---------------------------------------------------------------+
| LS length byte |
+---------------------------------------------------------------+
SECS List
A SECS list is a Perl list (array) based upon the SEMI E5-94 SECSII format. The list consists consecutive items where each item takes two positions in the list: the item header (IH) and the item body (IB). The item headers are always even number indices where the item bodies are odd number indices. The IH is a format code as specified in the Table 1 Item Format Codes unpack column. The IB for each format code is as follows:
- L
-
unpacked number
- S U F T
-
either scalarized array of numbers packed in accordance with SEMI E5-97 or a vectorized reference to an array of numbers
- B A J
-
unpacked string
The first item of a SECS list is always a SECS list Format Code wit a IH of U1 and a packed IB of either 'P' or 'S' depending upon whether the SECS list has information necessary to convert to Perl data structure, 'P', or not, 'S'.
arrayify subroutine
@array = arrayify( @var );
The purpose of the arrayify
subroutine is to provide a canoncial array representation of Perl reference types. When $var
is not a reference arrayify
subroutine passes $var
through unchanged; otherewise the ref($var) is changed to a reference to a canoncial array where the first member is the the $var
class, the second member the underlying data type. If ref($var) and the underlying type type are the same, then $var
is classless and the first member is the empty string ''. The rest of the members of the canonical array, based on the underlying data type, are as follows:
- 'HASH'
-
hash key, value pairs, sorted by the key
- 'ARRAY'
-
members of the array
- 'SCALAR'
-
the scalar
- 'REF'
-
the reference
- 'GLOB'
-
values of the
GLOB
in the following order:*$var{SCALAR}, *$var{ARRAY}, *$var{HASH}, *$var{CODE}, *$var{IO}, *$var{NAME}, *$var{PACKAGE}, "*$var"
itemify subroutine
$string = itemify($format, @elements);
$string = itemify($format, @elements, [@options]);
$string = itemify($format, @elements, {options});
The itemify
subroutine is the low-level work horse for the secsify
subroutine that produces a SEMI SECSII item $string
from a Perl SECS list item header $format
and item body @elements
.
For {type => 'binary'}, $string is a complete packed SEMI E5-94 SECII item. For {type => 'ascii'} or no type option, the $string
is the ascii unpacked SECSII item.
An unpacked SECSII item consists of the unpacked format code from the Table 1 Item Format Codes, the number of elements in the item body enclosed in brackets, followed by the elements in the item body. In accordance with SEMI E5-94, section 6, there will be no elements for format code 'L' or an element length of 0.
In case of an error, the return is an reference a error message.
listify subroutine
\@sec_list = listify(@vars);
The listify subroutine takes a list of Perl variables, @arg
that may contain references to nested data and converts it to a <SECS list that mimics a SECSII data structure of a linearized list of items.
Information is included to recontruct Perl hashes, arrays and objects by provided two item header for each Perl data type. The first item is the object class which is empty for Perl hashes and arrays and the second item is the Perl underlying data type. Valid Perl underlying data types are: HASH ARRAY SCALAR REF GLOB.
The return is either a reference to a SECS list or case of an error an error message. To determine an error from a SECS list , check if the return is a reference or a reference to an ARRAY.
neuterify subroutine
\@sec_list = neuterify($binary_secs);
\@sec_list = neuterify($binary_secs, @options);
\@sec_list = neuterify($binary_secs, [@options]);
\@sec_list = neuterify($binary_secs, {@options});
The neuterify
subroutine takes produces a @sec_list
from a SEMI E5-94 packed data structure $binary_secs
and produces a @sec_list
.
The neuterify
subroutine uses option {format => 'P'}, or {format => 'S'} as the value for the leading SECS list U1 format byte. SEMI E5-94 SECII item.
The return is either a reference to a SECS list or case of an error an error message. To determine an error from a SECS list , check if the return is a reference or a reference to an ARRAY.
scalarize subroutine
$error = scalarize( \@sec_list );
The scalarize
subroutine ensures that all the bodies in a SECS list for numeric items, format U, S, F, T, are scalar strings packed in accordance with SEMI E5-97.
secsify subroutine
$ascii_secs = secsify( \@sec_list);
$ascii_secs = secsify( \@sec_list, @options);
$ascii_secs = secsify( \@sec_list, [@options]);
$ascii_secs = secsify( \@sec_list, {@options});
$binary_secs = secsify( \@sec_list, type => 'binary');
$binary_secs = secsify( \@sec_list, type => 'binary', @options);
$binary_secs = secsify( \@sec_list, [type => 'binary',@options]);
$binary_secs = secsify( \@sec_list, {type => 'binary',@options});
The secsify
subroutine/method walks a data structure and converts all underlying array and hash references to arrays by applying the 'arrayify' subroutine/method.
In this module, there are only three Perlified SECSII elements that will be listified into SECSII message as follows:
OBJECT, INDEX OBJECT, and SCALAR
OBJECT => 'L', $number-of-elements,
'A', $class,
'A', $built-in-class,
@elements
@elements may contain a Perlified OBJECT, REFERENCE or SCALAR)
INDEX OBJECT => 'L' '2', 'A' 'Index', 'U4', $number-of-indices, @indices
(reference is index into the nested list of lists)
SCALAR = 'A', $scalar (Perl built-in class)
stringify subroutine
The stringify
subroutined stringifies a Perl data structure by applying the listify
and secify
subroutines.
transify subroutine
\@sec_list = transify($acsii_secs);
\@sec_list = transify($acsii_secs, @options);
\@sec_list = transify($acsii_secs, [@options]);
\@sec_list = transify($acsii_secs, {@options});
The transify
subroutine takes a free style text consisting of list of secsii items and converts it to SECS list. The transify
subroutine is very liberal in what it accepts as valid input.
The number of body elements may be supplied either as enclosed in brackets of a "comma" after the unpacked format code. Text strings may be enclosed in parentheses, brackets, or any other character.
The enclosing ending character may be escaped with the backslash '\'. List may be counted by suppling a count in either brackets or following a comma after the 'L' format character or by enclosing parentheseses, bracketers or any other character.
The transify
subroutine uses option {format => 'P'}, or {format => 'S'} as the value for the leading SECS list U1 format byte. SEMI E5-94 SECII item.
The return is either a reference to a SECS list or case of an error an error message. To determine an error from a SECS list , check if the return is a reference or a reference to an ARRAY.
vectorize subroutine
$error = vectorize( \@sec_list );
The vectorize
subroutine ensures that all the bodies in a SECS list for numeric items, format U, S, F, T, are references to an array of numbers.
REQUIREMENTS
The requirements are coming.
DEMONSTRATION
~~~~~~ Demonstration overview ~~~~~
Perl code begins with the prompt
=>
The selected results from executing the Perl Code follow on the next lines. For example,
=> 2 + 2
4
~~~~~~ The demonstration follows ~~~~~
=> use File::Package;
=> my $fp = 'File::Package';
=> use Data::Secs2 qw(arrayify itemify listify neuterify scalarize secsify
=> stringify transify vectorize);
=> my $uut = 'Data::Secs2';
=> my $loaded;
=> $uut->import( 'stringify' )
=> stringify( 'string' )
'string'
=> stringify( 2 )
2
=> stringify( '2', 'hello', 4 )
'U1[1] 80
U1[1] 2
A[5] hello
U1[1] 4
'
=> stringify( ['2', 'hello', 4] )
'U1[1] 80
L[5]
A[0]
A[5] ARRAY
U1[1] 2
A[5] hello
U1[1] 4
'
=> stringify( {header => 'To: world', body => 'hello'})
'U1[1] 80
L[6]
A[0]
A[4] HASH
A[4] body
A[5] hello
A[6] header
A[9] To: world
'
=> secsify( listify( ['2', 'hello', 4] ) )
'U1[1] 80
L[5]
A[0]
A[5] ARRAY
U1[1] 2
A[5] hello
U1[1] 4
'
=> secsify( listify( {header => 'To: world', body => 'hello'}) )
'U1[1] 80
L[6]
A[0]
A[4] HASH
A[4] body
A[5] hello
A[6] header
A[9] To: world
'
=> secsify( listify( '2', ['hello', 'world'], 512 ) )
'U1[1] 80
U1[1] 2
L[4]
A[0]
A[5] ARRAY
A[5] hello
A[5] world
U2[1] 512
'
=> my $obj = bless { To => 'nobody', From => 'nobody'}, 'Class::None'
=> secsify( listify( '2', { msg => ['hello', 'world'] , header => $obj } ) )
'U1[1] 80
U1[1] 2
L[6]
A[0]
A[4] HASH
A[6] header
L[6]
A[11] Class::None
A[4] HASH
A[4] From
A[6] nobody
A[2] To
A[6] nobody
A[3] msg
L[4]
A[0]
A[5] ARRAY
A[5] hello
A[5] world
'
=> secsify( listify( {msg => ['hello', 'world'] , header => $obj },
=> {msg => [ 'body' ], header => $obj} ) )
'U1[1] 80
L[6]
A[0]
A[4] HASH
A[6] header
L[6]
A[11] Class::None
A[4] HASH
A[4] From
A[6] nobody
A[2] To
A[6] nobody
A[3] msg
L[4]
A[0]
A[5] ARRAY
A[5] hello
A[5] world
L[6]
A[0]
A[4] HASH
A[6] header
L[2]
A[5] Index
U1[3] 2 0 3
A[3] msg
L[3]
A[0]
A[5] ARRAY
A[4] body
'
=> my $big_secs2 = unpack('H*',secsify( listify( ['2', 'hello', 4] ), {type => 'binary'}))
'a501500105410041054152524159a50102410568656c6c6fa50104'
=> secsify(neuterify (pack('H*',$big_secs2)))
'U1[1] 80
L[5]
A[0]
A[5] ARRAY
U1[1] 2
A[5] hello
U1[1] 4
'
=> my $ascii_secsii =
=> '
=> L
=> (
=> A \'\' A \'HASH\' A \'header\'
=> L [ A "Class::None" A "HASH"
=> A "From" A "nobody"
=> A "To" A "nobody"
=> ]
=> A "msg"
=> L,4 A[0] A[5] ARRAY
=> A "hello" A "world"
=> )
=> L
=> (
=> A[0] A "HASH" A /header/
=> L[2] A \'Index\' U1 2 0 3
=> A \'msg\'
=> L < A[0] A \'ARRAY\' A \'body\' >
=> )
=> '
=> my $list = transify ($ascii_secsii, format_code => 'P');
=> ref($list)
'ARRAY'
=> ref($list) ? secsify( $list ) : ''
'U1[1] 80
L[6]
A[0]
A[4] HASH
A[6] header
L[6]
A[11] Class::None
A[4] HASH
A[4] From
A[6] nobody
A[2] To
A[6] nobody
A[3] msg
L[4]
A[0]
A[5] ARRAY
A[5] hello
A[5] world
L[6]
A[0]
A[4] HASH
A[6] header
L[2]
A[5] Index
U1[3] 2 0 3
A[3] msg
L[3]
A[0]
A[5] ARRAY
A[4] body
'
=> ref(my $number_list = listify( [ [78,45,25], [512,1024], 100000] ))
'ARRAY'
=> secsify($number_list)
'U1[1] 80
L[5]
A[0]
A[5] ARRAY
U1[3] 78 45 25
U2[2] 512 1024
U4[1] 100000
'
=> vectorize($number_list)
''
=> [@{$number_list->[9]}]
[
78,
45,
25
]
=> [@{$number_list->[11]}]
[
512,
1024
]
=> [@{$number_list->[13]}]
[
100000
]
=> scalarize($number_list)
''
=> unpack('H*', $number_list->[9])
'4e2d19'
=> unpack('H*', $number_list->[11])
'02000400'
=> unpack('H*', $number_list->[13])
'000186a0'
QUALITY ASSURANCE
Running the test script 'Secs2.t' found in the "Data-Secs2-$VERSION.tar.gz" distribution file verifies the requirements for this module.
All testing software and documentation stems from the Software Test Description (STD) program module 't::Data::Secs2', found in the distribution file "Data-Secs2-$VERSION.tar.gz".
The 't::Data::Secs2' STD POD contains a tracebility matix between the requirements established above for this module, and the test steps identified by a 'ok' number from running the 'Secs2.t' test script.
The t::Data::Secs2' STD program module '__DATA__' section contains the data to perform the following:
to generate the test script 'Secs2.t'
generate the tailored STD POD in the 't::Data::Secs2' module,
generate the 'Secs2.d' demo script,
replace the POD demonstration section herein with the demo script 'Secs2.d' output, and
run the test script using Test::Harness with or without the verbose option,
To perform all the above, prepare and run the automation software as follows:
Install "Test_STDmaker-$VERSION.tar.gz" from one of the respositories only if it has not been installed:
http://www.softwarediamonds/packages/
http://www.perl.com/CPAN-local/authors/id/S/SO/SOFTDIA/
manually place the script tmake.pl in "Test_STDmaker-$VERSION.tar.gz' in the site operating system executable path only if it is not in the executable path
place the 't::Data::Secs2' at the same level in the directory struture as the directory holding the 'Data::Secs2' module
execute the following in any directory:
tmake -test_verbose -replace -run -pm=t::Data::Secs2
NOTES
FILES
The installation of the "Data-Secs2-$VERSION.tar.gz" distribution file installs the 'Docs::Site_SVD::Data_Secs2' SVD program module.
The __DATA__ data section of the 'Docs::Site_SVD::Data_Secs2' contains all the necessary data to generate the POD section of 'Docs::Site_SVD::Data_Secs2' and the "Data-Secs2-$VERSION.tar.gz" distribution file.
To make use of the 'Docs::Site_SVD::Data_Secs2' SVD program module, perform the following:
install "ExtUtils-SVDmaker-$VERSION.tar.gz" from one of the respositories only if it has not been installed:
http://www.softwarediamonds/packages/
http://www.perl.com/CPAN-local/authors/id/S/SO/SOFTDIA/
manually place the script vmake.pl in "ExtUtils-SVDmaker-$VERSION.tar.gz' in the site operating system executable path only if it is not in the executable path
Make any appropriate changes to the __DATA__ section of the 'Docs::Site_SVD::Data_Secs2' module. For example, any changes to 'Data::Secs2' will impact the at least 'Changes' field.
Execute the following:
vmake readme_html all -pm=Docs::Site_SVD::Data_Secs2
AUTHOR
The holder of the copyright and maintainer is
<support@SoftwareDiamonds.com>
COPYRIGHT NOTICE
Copyrighted (c) 2002 Software Diamonds
All Rights Reserved
BINDING REQUIREMENTS NOTICE
Binding requirements are indexed with the pharse 'shall[dd]' where dd is an unique number for each header section. This conforms to standard federal government practices, US DOD 490A 3.2.3.6. In accordance with the License, Software Diamonds is not liable for any requirement, binding or otherwise.
LICENSE
Software Diamonds permits the redistribution and use in source and binary forms, with or without modification, provided that the following conditions are met:
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
SOFTWARE DIAMONDS, http::www.softwarediamonds.com, PROVIDES THIS SOFTWARE 'AS IS' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL SOFTWARE DIAMONDS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING USE OF THIS SOFTWARE, EVEN IF ADVISED OF NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE POSSIBILITY OF SUCH DAMAGE.
3 POD Errors
The following errors were encountered while parsing the POD:
- Around line 1210:
Nested L<> are illegal. Pretending inner one is X<...> so can continue looking for other errors.
L<> starts or ends with whitespace
- Around line 1234:
Nested L<> are illegal. Pretending inner one is X<...> so can continue looking for other errors.
L<> starts or ends with whitespace
- Around line 1323:
Nested L<> are illegal. Pretending inner one is X<...> so can continue looking for other errors.
L<> starts or ends with whitespace