NAME

CAPE::Utils - A helpful library for with CAPE.

VERSION

Version 2.9.0

SYNOPSIS

Quick summary of what the module does.

Perhaps a little code snippet.

use CAPE::Utils;

my $cape_util=CAPE::Utils->new();

my $sub_results=$cape_util->submit(items=>@to_detonate,unique=>0, quiet=>1);
use JSON;
print encode_json($sub_results)."\n";

METHODS

new

Initiates the object. One argument is taken and that is the path to the INI config file. The default is '/usr/local/etc/cape_utils.ini' and if not found, the defaults will be used.

my $cape_util=CAPE::Utils->new('/path/to/some/config.ini');

connect

Return a DBH from DBI->connect for the CAPE SQL server.

This will die with the output from $DBI::errstr if it fails.

my $dbh = $cape->connect;

fail

Set one or more pending tasks to failed as below.

UPDATE tasks SET status = 'failed_processing' WHERE status = 'pending'

The following options are also supported.

- where :: Additional SQL where statements to add. Something must
           be specified for this, unless fail_all in the config is
           true. Otherwise this method will die.

my $rows=$cape_util->fail( where=>"target like '%foo%'");

get_pending_count

Get pending count pending tasks.

- where :: And additional where part of the SQL statement. "and" will
           automatically be used to join it with the rest of the
           statement. May not contain a ';'.
    - Default :: undef

my $count=$cape_util->get_pending_count;

get_pending

Returns a arrah ref of hash refs of rows from the tasks table where the status is set to pending via "select * from tasks where status = 'pending'"

- where :: And additional where part of the SQL statement. "and" will
           automatically be used to join it with the rest of the
           statement. May not contain a ';'.
    - Default :: undef

- where :: Additional SQL where statements to add.

get_pending_table

Generates a ASCII table for pending.

The following config variables can are relevant to this and may be overriden.

table_border
table_color
pending_columns
pending_target_clip
pending_time_clip

The following options are also supported.

- where :: And additional where part of the SQL statement. "and" will
           automatically be used to join it with the rest of the
           statement. May not contain a ';'.
    - Default :: undef

print $cape_util->get_pending_table( pending_columns=>'id,package');

get_running

Returns a array ref of hash refs of rows from the tasks table where the status is set to pending.

select * from tasks where status = 'running'

The statement above is used to find running tasks.

- where :: And additional where part of the SQL statement. "and" will
           automatically be used to join it with the rest of the
           statement. May not contain a ';'.
    - Default :: undef

use Data::Dumper;

my $running=$cape_utils->get_running;
print Dumper($running);

get_running_count

Get pending count running tasks.

select * from tasks where status = 'running'

The statement above is used to find running tasks.

- where :: And additional where part of the SQL statement. "and" will
           automatically be used to join it with the rest of the
           statement. May not contain a ';'.
    - Default :: undef

my $count=$cape_util->get_running_count;

get_running_table

Generates a ASCII table for pending.

The following config variables can are relevant to this and may be overriden.

table_border
table_color
running_columns
running_target_clip
running_time_clip

The statement below is used to find running tasks.

select * from tasks where status = 'running'

The following options are also supported.

- where :: And additional where part of the SQL statement. "and" will
           automatically be used to join it with the rest of the
           statement. May not contain a ';'.
    - Default :: undef

print $cape_util->get_pending_table( pending_columns=>'id,package');

get_tasks

Returns a array ref of hash refs of rows from the tasks table where the status is set to pending.

- where :: The where part of the SQL statement. May not contain a ';'.
    - Default :: undef

- order :: Column to order by.
    - Default :: id

- limit :: Number of items to return.
    - Default :: 100

- direction :: Direction to order in.
    - Default :: desc

use Data::Dumper;

A small example showing getting running, ordering by category, and limiting to 20.

my $tasks=$cape_utils->get_tasks(where=>"status = 'running'", limit=>20, order=>"category", direction=>'desc');
print Dumper($running);

get_tasks_count

Gets a count of tasks.

- where :: The where part of the SQL statement. May not contain a ';'.
    - Default :: undef

use Data::Dumper;

A small example showing getting running, ordering by category, and limiting to 20.

my $count=$cape_util->get_tasks_count(where=>"status = 'running'", limit=>20, order=>"category", direction=>'desc');

get_tasks_table

Generates a ASCII table for tasks.

The following config variables can are relevant to this and may be overriden.

table_border
table_color
task_columns
task_target_clip
task_time_clip

The following options are also supported.

- where :: Additional SQL where statements to add.
    - Default :: undef

- order :: Column to order by.
    - Default :: id

- limit :: Number of items to return.
    - Default :: 100

- direction :: Direction to order in.
    - Default :: desc

print $cape_util->get_tasks_table( where => "status = 'reported'");

munge

Munges the specified report file.

$cape_utils->munge(file=>$report_file);

search

Searches the list of tasks. By default everything will be return ed.

- where :: Additional SQL where statements to use for searching.
           May not contain a ';'.
  - Default :: undef

Addtionally there are also helpers for searching. These may not contain either a /\'/ or a /\\/. They will be joined via and.

The following are handled as a simple equality.

- timeout
- memory
- enforce_timeout
- timedout

The following are numeric. Each one may accept multiple comma sperated values. The equalities =, >,>=, <=, and ! are supported. If no equality is specified, then = is used.

- id
- timeout
- priority
- dropped_files
- running_processes
- api_calls
- domains
- signatures_total
- signatures_alert
- files_written
- registry_keys_modified
- crash_issues
- anti_issues
- sample_id
- machine_id

# will result in id >= 3 and id < 44
id => '>=3,<44'

# either of these will be id = 4
id => '=4'
id => '4'

The following are string items. As is, they are evaluated as a simple equality. If ending with ending in '_like', they will be evaluated as a like.

- target
- category
- custom
- machine
- package
- route
- tags_tasks
- options
- platform

# becomes... target = 'foo'
target => 'foo'

# becomes... target like 'foo%'
target_like => 'foo%'

submit

Submits files to CAPE.

- clock :: Timestamp to use for setting the clock to of the VM for
  when executing the item. If left undefined, it will be
  autogenerated.
  - Format :: mm-dd-yyy HH:MM:ss

- items :: A array ref of items to submit. If a directory is listed in
  here, it will be read, but subdirectories will not be recursed. They
  will be ignored.

- name_regex :: Regex to use for matching items in a submitted dir.
  Only used if the a submitted item is a dir.
  - Default :: undef

- mime_regex :: Array ref of desired mime types to match via
  regex. Only used if the a submitted item is a dir.
  - Default :: undef

- timeout :: Value to use for timeout. Set to 0 to not enforce.
  - Default :: 200

- machine :: The machine to use for this. If not defined, first
  available will be used.
  - Default :: undef

- package :: Package to use, if not letting CAPE decide.
  - Default :: undef

- options :: Option string to be passed via --options.
  - Default :: undef

- random :: If it should randomize the order of submission.
  - Default :: 1

- tags :: Tags to be passed to the script via --tags.
  - Default :: undef

- platform :: What to pass to --platform.
  - Default :: undef

- custom :: Any custom values to pass via --custom.
  - Default :: undef

- enforce_timeout :: Force it to run the entire period.
  - Default :: 0

- unique :: Only submit it if it is unique.
    - Default :: 0

-quiet :: Do not print the results.
    - Default :: 0

The retuned value is a hash ref where the keys are successfully submitted files and values of those keys is the task ID.

my $sub_results=$cape_util->submit(items=>@to_detonate,unique=>0, quiet=>1);
use JSON;
print encode_json($sub_results)."\n";

timestamp

Creates a timestamp to be used with utils/submit. localtime is used to get the current time.

print $cape_util->timestamp."\n";

shuffle

Performa a Fisher Yates shuffle on the passed array ref.

check_remote

Checks the remote connection.

Two variablesare required, API key and IP.

$results=$cape_utils->check_remote(apikey=>$apikey, remote=>$remote_ip);
if (!$results){
    print "unauthed\n";
    return;
}

eve_process

Process the finished tasks for CAPEv2.

$cape_utils->eve_process;

CONFIG FILE

The default config file is '/usr/local/etc/cape_utils.ini'.

The defaults are as below, which out of the box, it will work by default with CAPEv2 in it's default config.

# The DBI dsn to use
dsn=dbi:Pg:dbname=cape
# DB user
user=cape
# DB password
pass=
# the install base for CAPEv2
base=/opt/CAPEv2/
# 0/1 if poetry should be used
poetry=1
# 0/1 if fail should be allowed to run with out a where statement
fail_all=0
# colums to use for pending table show
pending_columns=id,target,package,timeout,ET,route,options,clock,added_on
# colums to use for runniong table show
running_columns=id,target,package,timeout,ET,route,options,clock,added_on,started_on,machine
# colums to use for tasks table
task_columns=id,target,package,timeout,ET,route,options,clock,added_on,latest,machine,status
# if the target column for running table display should be clipped to the filename
running_target_clip=1
# if microseconds should be clipped from time for running table display
running_time_clip=1
# if the target column for pending table display should be clipped to the filename
pending_target_clip=1
# if microseconds should be clipped from time for pending table display
pending_time_clip=1
# if the target column for task table display should be clipped to the filename
task_target_clip=1
# if microseconds should be clipped from time for task table display
task_time_clip=1
# default table color
table_color=Text::ANSITable::Standard::NoGradation
# default table border
table_border=ASCII::None
# when submitting use now for the current time
set_clock_to_now=1
# default timeout value for submit
timeout=200
# default value for enforce timeout for submit
enforce_timeout=0
# how to auth for mojo_cape_submit
# ip = match against subnets
# apikey = use apikey
# both = require both to match
# either = either may work
auth=ip
# the api key to for with mojo_cape_submit
#apikey=
# comma seperated list of allowed subnets for mojo_cape_submit
subnets=192.168.0.0/16,127.0.0.1/8,::1/128,172.16.0.0/12,10.0.0.0/8
# incoming dir to use for mojo_cape_submit
incoming=/malware/client-incoming
# directory to store json data files for submissions recieved by mojo_cape_submit
# this directory is also used for storing run specific eves
incoming_json=/malware/incoming-json
# Location to write the eve log to.
eve=/opt/CAPEv2/log/eve.json
# how far to go back for processing eve
eve_look_back=360
# malscore for changing the event_type for eve from potential_malware_detonation to alert
malscore=0

Report Munge Section

INI sections matching /^munge\_/ will be used for report munging. This requires two values for that sections, 'check' and 'munge'.

'check' is a path to a Perl script that will wrapped in a eval and require to check if the file should be munged or not.

'munge' is a path to a Perl script that will wrapped in a eval and require to do the munging.

Below is a example showing the setup for a single script.

[munge_pdf]
check=/usr/local/etc/cape_utils_munge/pdf_check
munge=/usr/local/etc/cape_utils_munge/pdf_munge

If more than one munge section exists, they are ran in sorted order.

If the paths specied do not start with a '/', './', or '../', then '/usr/local/etc/cape_utils_munge/' is applied to the start.

The scripts are read as evaled strings.

The relevant variables are as below.

- $munge_it :: Perl boolean for if it should be munged or not. Should be set by the check script.

- $report :: The hash ref containing the parsed JSON report.

- $changed :: Perl boolean for if it changed or not.

For some examples see the directory 'munge_examples'.

CAPEv2 lite.json to EVE handling

Tasks are found by looking back X number of seconds in the tasks table for tasks that have reported. The amount of time is determined by the config value 'eve_look_back'.

It will check if a task has been processed already or not be seeing if a task specified EVE JSON has been created under the 'incoming_json' directory. This is in the format $task_id.'eve.json'. If not, it will proceed.

It reads the 'lite.json' report for task as well as the incoming JSON. It then copies the keys 'signatures' and 'malscore' into the hash for the incoming JSON and writes it out to $task_id.'eve.json' and appending it to the file specified via the config value 'eve'.

The are two possible values for 'event_type', 'potential_malware_detonation' and 'alert'. 'potential_malware_detonation' is changed to alert when 'malscore' goves over the value specified via config value 'malscore'.

'row' is the full row for the task in question from the task table as a hash.

'signatures' is copied from '.signature' in the report JSON.

AUTHOR

Zane C. Bowers-Hadley, <vvelox at vvelox.net>

BUGS

Please report any bugs or feature requests to bug-cape-utils at rt.cpan.org, or through the web interface at https://rt.cpan.org/NoAuth/ReportBug.html?Queue=CAPE-Utils. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

perldoc CAPE::Utils

You can also look for information at:

RT: CPAN's request tracker (report bugs here)

https://rt.cpan.org/NoAuth/Bugs.html?Dist=CAPE-Utils
Search CPAN

https://metacpan.org/release/CAPE-Utils
Git

"CAPE-Utils.git" in git@github.com:VVelox
Web

https://github.com/VVelox/CAPE-Utils

ACKNOWLEDGEMENTS

LICENSE AND COPYRIGHT

This is free software, licensed under:

The Artistic License 2.0 (GPL Compatible)

To install CAPE::Utils, copy and paste the appropriate command in to your terminal.

cpanm

cpanm CAPE::Utils

CPAN shell

perl -MCPAN -e shell
install CAPE::Utils

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

VERSION

SYNOPSIS

METHODS

new

connect

fail

get_pending_count

get_pending

get_pending_table

get_running

get_running_count

get_running_table

get_tasks

get_tasks_count

get_tasks_table

munge

search

submit

timestamp

shuffle

check_remote

eve_process

CONFIG FILE

Report Munge Section

CAPEv2 lite.json to EVE handling

AUTHOR

BUGS

SUPPORT

ACKNOWLEDGEMENTS

LICENSE AND COPYRIGHT

Module Install Instructions