clients Package¶

`hemlock_base` Module¶

This module is the main controller code for running clients that sit in this directory.

Created on 19 August 2013 @author: Charlie Lewis

class hemlock.clients.hemlock_base.Hemlock_Base[source]¶

This class is responsible for validating clients and controlling the orchestration between clients the Hemlock metadata/data store.

client_import(debug, client)[source]¶

Imports the client specific as a python module.

Parameters:	debug – instance of `Hemlock_Debugger` client – string containing the name of the technology of the client i.e. mysql
Returns:	returns string name of client credential file and instance of the client class

connect_server(debug, server_dict, no_couchbase)[source]¶

Connects to the Hemlock couchbase server.

Parameters:	debug – instance of `Hemlock_Debugger` server_dict – credentials for connecting to the Hemlock server to be able to verify the client system no_couchbase – flag of whether or not to use a couchbase connection or an elasticsearch connection
Returns:	returns an instance of the couchbase connection

get_args(debug)[source]¶

Gets the arguments from the command line.

Parameters:	debug – instance of `Hemlock_Debugger`
Returns:	returns list of arguments

get_creds(debug, CLIENT_CREDS_FILE)[source]¶

Gets the credentials for connecting the client and the credentials for connecting to the Hemlock server.

Parameters:	debug – instance of `Hemlock_Debugger` CLIENT_CREDS_FILE – path to file containing the client credentials
Returns:	returns two dictionaries of the client and server credentials

print_help()[source]¶: Prints out help for the hemlock_base class.

process_args(debug, args)[source]¶

Processes the arguments passed in to ensure that the right ones are supplied before trying to execute against them.

Parameters:	debug – instance of `Hemlock_Debugger` args – list of arguments that are passed in
Returns:	returns client system uuid, the client technology to use, and the number of splits (defaults to -1 if not supplied)

send_data(debug, data_list, desc_list, h_server, client_uuid, no_couchbase)[source]¶

Sends data to the Hemlock couchbase server that is recieved from the client system.

Parameters:

debug – instance of Hemlock_Debugger
data_list – array of arrays containing data from the client
desc_list – list containing a corresponding schema to the data, can be empty
h_server – instnace of the couchbase connection
client_uuid – uuid of the client system
no_couchbase – flag of whether or not to use a couchbase connection or an elasticsearch connection

stream_callback(data)[source]¶

Callback for hstream_odd, should only happen if something failed.

Parameters:	data – data that failed

stream_workers(debug)[source]¶

Spawns asyncronous workers when calling an hstream_odd client.

Parameters:	debug – instance of `Hemlock_Debugger`

update_hemlock(debug, client_uuid, server_dict)[source]¶

Sends data to the Hemlock couchbase server that is recieved from the client system.

Parameters:	debug – instance of `Hemlock_Debugger` client_uuid – uuid of the client system server_dict – credentials for connecting to the Hemlock server to be able to verify the client system

verify_system(debug, client_uuid, server_dict)[source]¶

Verifies that the system supplied exists in the Hemlock system.

Parameters:	debug – instance of `Hemlock_Debugger` client_uuid – uuid of the client system that is being verified server_dict – credentials for connecting to the Hemlock server to be able to verify the client system

`hemlock_debugger` Module¶

This module is used for running a debugger across all modules in Hemlock.

Create on 28 August 2013 @author: Charlie Lewis

class hemlock.clients.hemlock_debugger.Hemlock_Debugger[source]¶

This class is responsible for printing out debug statements as well as write them to file.

debug(debug, string)[source]¶

Prints and logs all debug statements.

Parameters:	debug – instance of `Hemlock_Debugger` string – stringified version of whatever is being debugged.

`hemlock_runner` Module¶

This module gets all needed credentials in preparation for getting data from clients and into Hemlock’s metadata/data store.

Created on 26 August 2013 @author: Charlie Lewis

class hemlock.clients.hemlock_runner.Hemlock_Runner[source]¶

This class is responsible for pulling out the credentials from the Hemlock server for both the client system and the Hemlock metadata/data store.

get_creds(debug, m_server, client_id, aes_key)[source]¶

Retrieves the credentials for both the client system as well as the Hemlock server that are stored in the Hemlock MySQL server.

Parameters:	debug – instance of `Hemlock_Debugger` m_server – instance of MySQL connection client_id – uuid of the client to get credentials for aes_key – AES key for decrypting the stored credentials
Returns:	returns two dictionaries for the client system and Hemlock server credentials

mysql_server(debug, server, user, pw, db)[source]¶

Connects to the Hemlock MySQL server to get an instance for retrieving credentials.

Parameters:	debug – instance of `Hemlock_Debugger` server – server address of the Hemlock MySQL server user – user account to connect to the Hemlock MySQL server pw – password of the user account db – database to connect to in the Hemlock MySQL server
Returns:	returns an instance of the mysql connection

`hfs` Module¶

class hemlock.clients.hfs.HFs[source]¶

connect_client(debug, client_dict)[source]¶

format_lists(debug, j_list, h_server, client_uuid, no_couchbase)[source]¶

get_data(debug, client_dict, c_server, h_server, client_uuid, no_couchbase)[source]¶

scan_file_types(debug, c_server, h_server, client_uuid, no_couchbase)[source]¶

`hfs_old` Module¶

class hemlock.clients.hfs_old.HFs[source]¶

connect_client(debug, client_dict)[source]¶

convert_pdf(debug, input)[source]¶

format_lists(debug, j_list, h_server, client_uuid, no_couchbase)[source]¶

get_data(debug, client_dict, c_server, h_server, client_uuid, no_couchbase)[source]¶

process_files(debug, input, h_server, client_uuid, no_couchbase)[source]¶

`hhbase` Module¶

`hhdfs` Module¶

`hmongo` Module¶

class hemlock.clients.hmongo.HMongo[source]¶

connect_client(debug, client_dict)[source]¶

get_data(debug, client_dict, c_server, h_server, client_uuid, no_couchbase)[source]¶

`hmysql` Module¶

class hemlock.clients.hmysql.HMysql[source]¶

connect_client(debug, client_dict)[source]¶

get_data(debug, client_dict, c_server, h_server, client_uuid, no_couchbase)[source]¶

`hredis` Module¶

class hemlock.clients.hredis.HRedis[source]¶

connect_client(debug, client_dict)[source]¶

get_data(debug, client_dict, c_server, h_server, client_uuid, no_couchbase)[source]¶

`hrest` Module¶

class hemlock.clients.hrest.HRest[source]¶

connect_client(debug, client_dict)[source]¶

get_data(debug, client_dict, c_server, h_server, client_uuid, no_couchbase)[source]¶

`hstream_odd` Module¶

class hemlock.clients.hstream_odd.HStream_Odd[source]¶

connect_client(debug, client_dict, h_server, client_uuid, no_couchbase)[source]¶

start(debug, hostname, port, h_server, client_uuid, no_couchbase)[source]¶

hemlock.clients.hstream_odd.handle(debug, connection, address, h_server, client_uuid, no_couchbase)[source]¶

Subpackages¶

file_types Package

clients Package¶

`hemlock_base` Module¶

`hemlock_debugger` Module¶

`hemlock_runner` Module¶

`hfs` Module¶

`hfs_old` Module¶

`hhbase` Module¶

`hhdfs` Module¶

`hmongo` Module¶

`hmysql` Module¶

`hredis` Module¶

`hrest` Module¶

`hstream_odd` Module¶

Subpackages¶

Table Of Contents

Previous topic

Next topic

This Page

Navigation

clients Package¶

hemlock_base Module¶

hemlock_debugger Module¶

hemlock_runner Module¶

hfs Module¶

hfs_old Module¶

hhbase Module¶

hhdfs Module¶

hmongo Module¶

hmysql Module¶

hredis Module¶

hrest Module¶

hstream_odd Module¶

Subpackages¶

Table Of Contents

Previous topic

Next topic

This Page

Quick search

Navigation

`hemlock_base` Module¶

`hemlock_debugger` Module¶

`hemlock_runner` Module¶

`hfs` Module¶

`hfs_old` Module¶

`hhbase` Module¶

`hhdfs` Module¶

`hmongo` Module¶

`hmysql` Module¶

`hredis` Module¶

`hrest` Module¶

`hstream_odd` Module¶