hemlock Package

hemlock Module

This module is the main core of Hemlock and interfaces with and controls the majority of other modules in this package.

Created on 19 August 2013 @author: Charlie Lewis

class hemlock.hemlock.Hemlock

This class is responsible for driving the API and the core functionality of Hemlock.

check_args(args, arg_d, var_d)

Checks arguments supplied.

Parameters:

• args – arguments to pass in from API
• arg_d – list of supplied arguments
• var_d – dictionary of key/values made from the arguments

Returns:

returns a dictionary of the arguments supplied

client_add_schedule(args, var_d)

Adds a specific schedule to a specific client.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

client_get(args, var_d)

Gets a specific client.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

client_list(args, var_d)

Lists all clients.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

client_purge(args, var_d)

Purges a specific client.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

client_remove_schedule(args, var_d)

Removes a specific schedule from a specific client.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

client_run(args, var_d)

Runs a client.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

client_schedule(args, var_d)

Schedules a client.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

client_schedules_list(args, var_d)

Lists all schedules assigned to a specific client.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

client_store(args, var_d)

Stores a client.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

client_systems_list(args, var_d)

Lists all systems assigned to a specific client.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

connect_server(debug, c_server, c_user, c_bucket, c_pw, es, no_couchbase)

Connects to the Hemlock couchbase server.

Parameters:

• debug – instance of Hemlock_Debugger
• c_server – couchbase server ip address
• c_user – couchbase username
• c_bucket – couchbase bucket
• c_pw – couchbase password
• es – elasticsearch server ip address
• no_couchbase – flag of whether or not to use a couchbase connection or an elasticsearch connection

Returns:

returns an instance of the couchbase or elasticsearch connection

deregister_local_system(args, var_d)

Deegisters a specific local system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

deregister_remote_system(args, var_d)

Deregisters a specific remote system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

get_auth()

Gets the authentication of supplied arguments, environment variables, or sets them to defaults.

Returns:returns each of the parsed out arguments

hemlock_server_store(args, var_d)

Stores Hemlock server credentials.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

list_all(args, var_d)

Lists everything that is stored.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

mysql_server(debug, server, user, pw, db)

Connects to the Hemlock MySQL Server

Parameters:

• debug – instance of Hemlock_Debugger
• server – server address of the Hemlock MySQL server
• user – user account to connect to the Hemlock MySQL server
• pw – password of the user account
• db – database to connect to in the Hemlock MySQL server

Returns:

returns an instance of the mysql connection

parse_auth()

Parses options for authentication if supplied as arguments.

Returns:returns an instance of PassThroughOptionParser

print_help(action)

Prints out help.

Parameters:action – list of API action parsed into elements

process_action(debug, action, var_d, m_server, c_server, c_user, bucket, c_pw, no_couchbase, es)

Processes the action that was supplied.

Parameters:

• debug – instance of Hemlock_Debugger
• action – action to be performed
• var_d – dictionary of key/values that contain the parameters for the action
• m_server – instance of MySQL connection
• c_server – couchbase server ip address
• c_user – couchbase username
• bucket – couchbase bucket
• c_pw – couchbase password
• no_couchbase – flag of whether or not to use a couchbase connection or an elasticsearch connection
• es – elasticsearch server ip address

Returns:

list of results and any errors that may have occurred.

process_args(debug, args)

Processes arguments by directing the supplied action to the proper function and ensure that all required fields are present.

Parameters:

• debug – instance of Hemlock_Debugger
• args – list of supplied arguments

Returns:

returns a dictionary of argument key/values and the API action

query_data(args, var_d)

Queries data stored in Hemlock filtering based on the authenticated user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

read_creds(debug)

Reads in the credentials file for the Hemlock system and stores them in the local environment variables of the shell.

Parameters:debug – instance of Hemlock_Debugger

register_local_system(args, var_d)

Registers a local system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

register_remote_system(args, var_d)

Registers a remote system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

role_create(args, var_d)

Creates a role.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

role_delete(args, var_d)

Deletes a specific role.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

role_get(args, var_d)

Gets a specific role.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

role_list(args, var_d)

Lists all roles.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

role_users_list(args, var_d)

Lists all users assigned to a specific role.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_add_client(args, var_d)

Adds a specific client to a specific schedule.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_change_server(args, var_d)

Changes the server that a specific schedule runs on.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_clients_list(args, var_d)

Lists all clients assigned to a specific schedule.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_delete(args, var_d)

Deletes a specific schedule.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_get(args, var_d)

Gets a specific schedule.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_list(args, var_d)

Lists all schedules.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_remove_client(args, var_d)

Removes a specific client from a specific schedule.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_server_create(args, var_d)

Creates a schedule server.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_server_delete(args, var_d)

Deletes a specific schedule server.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_server_get(args, var_d)

Gets a specific scheduler server.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

schedule_server_list(args, var_d)

Lists all schedule servers.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

start_scheduler(args, var_d)

Starts the scheduler daemon.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

system_add_tenant(args, var_d)

Adds a specific tenant to a specific system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

system_clients_list(args, var_d)

Lists all clients assigned to a specific system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

system_get(args, var_d)

Gets a specific system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

system_list(args, var_d)

Lists all clients assigned to a specific system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

system_remove_tenant(args, var_d)

Removes a specific tenant from a specific system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

system_tenants_list(args, var_d)

Lists all tenants assigned to a specific system.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

tenant_create(args, var_d)

Creates a tenant.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

tenant_delete(args, var_d)

Deletes a specific tenant.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

tenant_get(args, var_d)

Gets a specific tenant.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

tenant_list(args, var_d)

Lists all tenants.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

tenant_systems_list(args, var_d)

Lists all systems assigned to a specific tenant.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

tenant_users_list(args, var_d)

Lists all users assigned to a specific tenant.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_add_role(args, var_d)

Adds a specific role to a specific user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_add_tenant(args, var_d)

Adds a specific tenant to a specific user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_create(args, var_d)

Creates a user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_delete(args, var_d)

Deletes a specific user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_get(args, var_d)

Gets a specific user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_list(args, var_d)

Lists all users.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_remove_role(args, var_d)

Removes a specific role from a specific user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_remove_tenant(args, var_d)

Removes a specific tenant from a specific user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_roles_list(args, var_d)

Lists all roles assigned to a specific user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

user_tenants_list(args, var_d)

Lists all tenants assigned to a specific user.

Parameters:

• args – arguments to pass in from API
• var_d – dictionary of key/values made from the arguments

Returns:

returns a list of the arguments supplied

hemlock_options_parser Module

class hemlock.hemlock_options_parser.PassThroughOptionParser(usage=None, option_list=None, option_class=<class optparse.Option at 0x20a7a78>, version=None, conflict_handler='error', description=None, formatter=None, add_help_option=True, prog=None, epilog=None)

Bases: optparse.OptionParser

An unknown option pass-through implementation of OptionParser.

When unknown arguments are encountered, bundle with largs and try again, until rargs is depleted.

sys.exit(status) will still be called if a known argument is passed incorrectly (e.g. missing arguments or bad argument types, etc.)

hemlock_scheduler Module

This module controlls and runs the scheduler for performing actions between client systems and the Hemlock server.

Created on 30 August 2013 @author: Charlie Lewis

class hemlock.hemlock_scheduler.Hemlock_Scheduler

This class is responsible for spawning and controlling the the scheduler and all operations that are scheduled in a cron-like fashion.

check_schedules()

Checks for existing schedules, cleans up ones that no longer need to run, starts new ones that need to be scheduled.

init_schedule()

Initialize the scheduler.

Returns:an instance of the scheduler.

job_work(server_dict, name)

Do the actual work that was scheduled at the scheduled tiem.

Parameters:

• server_dict – dictionary of server credentials
• name – uuid of the client

schedule_job(function, periodicity, start_time)

Schedule a new job.

Parameters:

• function – function to be called that does the work
• periodicity – how often to run the scheduled work
• start_time – when to start the job

schedule_job_cron(function, server_dict, name, minute, hour, day_of_month, month, day_of_week)

Schedule a new cron job.

Parameters:

• function – function to be called that does the work
• server_dict – dictionary of server credentials
• name – name of the job
• minute – cron minute to run the job
• hour – cron hour to run the job
• day_of_month – cron day_of_month to run the job
• month – cron month to run the job
• day_of_week – cron day_of_week to run the job

test_hemlock Module

Test module for hemlock.py

Created on 19 August 2013 @author: Charlie Lewis

class hemlock.test_hemlock.TestClass

Test class for hemlock.py

connect_mysql(debug, server, user, pw, db)

Makes a connection to the test Hemlock MySQL server.

Returns:returns an instance of the MySQL connection

process_client_get()

Tests client-get action.

Returns:returns any data and a list of any errors

process_client_list()

Tests client-list action.

Returns:returns any data and a list of any errors

process_client_purge()

Tests client-purge action.

Returns:returns any data and a list of any errors

process_client_run()

Tests client-run action.

Returns:returns any data and a list of any errors

process_client_schedule()

Tests client-schedule action.

Returns:returns any data and a list of any errors

process_client_store()

Tests client-store action.

Returns:returns any data and a list of any errors

process_deregister_local_system()

Tests deregister-local-system action.

Returns:returns any data and a list of any errors

process_deregister_remote_system()

Tests deregister-remote-system action.

Returns:returns any data and a list of any errors

process_list_all()

Tests list-all action.

Returns:returns any data and a list of any errors

process_query_data()

Tests query-data action.

Returns:returns any data and a list of any errors

process_register_local_system()

process_register_remote_system()

Tests register-remote-system action.

Returns:returns any data and a list of any errors

process_role_create()

Tests role-create action.

Returns:returns any data and a list of any errors

process_role_delete()

Tests role-delete action.

Returns:returns any data and a list of any errors

process_role_list()

Tests role-list action.

Returns:returns any data and a list of any errors

process_role_users_list()

Tests role-users-list action.

Returns:returns any data and a list of any errors

process_schedule_change_server()

Tests schedule-change-server action.

Returns:returns any data and a list of any errors

process_schedule_get()

Tests schedule-get action.

Returns:returns any data and a list of any errors

process_schedule_list()

Tests schedule-list action.

Returns:returns any data and a list of any errors

process_schedule_server_create()

Tests schedule-server-create action.

Returns:returns any data and a list of any errors

process_schedule_server_delete()

Tests schedule-server-delete action.

Returns:returns any data and a list of any errors

process_schedule_server_get()

Tests schedule-server-get action.

Returns:returns any data and a list of any errors

process_schedule_server_list()

Tests schedule-server-list action.

Returns:returns any data and a list of any errors

process_start_scheduler()

Tests start-scheduler action.

Returns:returns any data and a list of any errors

process_system_add_tenant()

Tests system-add-tenant action.

Returns:returns any data and a list of any errors

process_system_get()

Tests system-get action.

Returns:returns any data and a list of any errors

process_system_list()

Tests system-list action.

Returns:returns any data and a list of any errors

process_system_remove_tenant()

Tests system-remove-tenant action.

Returns:returns any data and a list of any errors

process_system_tenants_list()

Tests system-tenants-list action.

Returns:returns any data and a list of any errors

process_tenant_create()

Tests tenant-create action.

Returns:returns any data and a list of any errors

process_tenant_delete()

Tests tenant-delete action.

Returns:returns any data and a list of any errors

process_tenant_get()

Tests tenant-get action.

Returns:returns any data and a list of any errors

process_tenant_list()

Tests tenant-list action.

Returns:returns any data and a list of any errors

process_tenant_systems_list()

Tests tenant-systems-list action.

Returns:returns any data and a list of any errors

process_tenant_users_list()

Tests tenant-users-list action.

Returns:returns any data and a list of any errors

process_user_add_role()

Tests user-add-role action.

Returns:returns any data and a list of any errors

process_user_add_tenant()

Tests user-add-tenant action.

Returns:returns any data and a list of any errors

process_user_create()

process_user_delete()

Tests user-delete action.

Returns:returns any data and a list of any errors

process_user_get()

Tests user-get action.

Returns:returns any data and a list of any errors

process_user_list()

Tests user-list action.

Returns:returns any data and a list of any errors

process_user_remove_role()

Tests user-remove-role action.

Returns:returns any data and a list of any errors

process_user_remove_tenant()

Tests user-remove-tenant action.

Returns:returns any data and a list of any errors

process_user_roles_list()

Tests user-roles-list action.

Returns:returns any data and a list of any errors

process_user_tenants_list()

Tests user-tenants-list action.

Returns:returns any data and a list of any errors

test_connect_mysql()

Calls the test function for connecting to MySQL.

test_process_client_get()

Calls the test function for the client-get action.

test_process_client_list()

Calls the test function for the client-list action.

test_process_client_purge()

Calls the test function for the client-purge action.

test_process_client_run()

Calls the test function for the client-run action.

test_process_client_schedule()

Calls the test function for the client-schedule action.

test_process_client_store()

Calls the test function for the client-store action.

test_process_deregister_local_system()

Calls the test function for the deregister-local-system action.

test_process_deregister_remote_system()

Calls the test function for the deregister-remote-system action.

test_process_list_all()

Calls the test function for list-all action.

test_process_query_data()

Calls the test function for the query-data action.

test_process_register_local_system()

Calls the test function for the register-local-system action.

test_process_register_remote_system()

Calls the test function for the register-remote-system action.

test_process_role_create()

Calls the test function for the role-create action.

test_process_role_delete()

Calls the test function for the role-delete action.

test_process_role_list()

Calls the test function for the role-list action.

test_process_role_users_list()

Calls the test function for the role-users-list action.

test_process_schedule_change_server()

Calls the test function for the schedule-change-server action.

test_process_schedule_get()

Calls the test function for the schedule-get action.

test_process_schedule_list()

Calls the test function for the schedule-list action.

test_process_schedule_server_create()

Calls the test function for the schedule-server-create action.

test_process_schedule_server_delete()

Calls the test function for the schedule-server-delete action.

test_process_schedule_server_get()

Calls the test function for the schedule-server-get action.

test_process_schedule_server_list()

Calls the test function for the schedule-server-list action.

test_process_start_scheduler()

Calls the test function for the start-scheduler action.

test_process_system_add_tenant()

Calls the test function for the system-add-tenant action.

test_process_system_get()

Calls the test function for the system-get action.

test_process_system_list()

Calls the test function for the system-list action.

test_process_system_remove_tenant()

Calls the test function for the system-remove-tenant action.

test_process_system_tenants_list()

Calls the test function for the system-tenants-list action.

test_process_tenant_create()

Calls the test function for the tenant-create action.

test_process_tenant_delete()

Calls the test function for the tenant-delete action.

test_process_tenant_get()

Calls the test function for the tenant-get action.

test_process_tenant_list()

Calls the test function for the tenant-list action.

test_process_tenant_systems_list()

Calls the test function for the tenant-systems-list action.

test_process_tenant_users_list()

Calls the test function for the tenant-users-list action.

test_process_user_add_role()

Calls the test function for the user-add-role action.

test_process_user_add_tenant()

Calls the test function for the user-add-tenant action.

test_process_user_create()

Calls the test function for the user-create action.

test_process_user_delete()

Calls the test function for the user-delete action.

test_process_user_get()

Calls the test function for the user-get action.

test_process_user_list()

Calls the test function for the user-list action.

test_process_user_remove_role()

Calls the test function for the user-remove-role action.

test_process_user_remove_tenant()

Calls the test function for the user-remove-tenant action.

test_process_user_roles_list()

Calls the test function for the user-roles-list action.

test_process_user_tenants_list()

Calls the test function for the user-tenants-list action.

Subpackages

• clients Package
  • hemlock_base Module
  • hemlock_debugger Module
  • hemlock_runner Module
  • hfs Module
  • hfs_old Module
  • hhbase Module
  • hhdfs Module
  • hmongo Module
  • hmysql Module
  • hredis Module
  • hrest Module
  • hstream_odd Module
  • Subpackages
    • file_types Package
      • hcsv Module
      • hdoc Module
      • hgeneric Module
      • hjson Module
      • hpcap Module
      • hpdf Module
      • hppt Module
      • htext Module
      • hxls Module
      • hxml Module
• load_scripts Package
  • csv_to_mysql Module
  • csv_to_redis_hash Module
  • doc_to_mongo Module
• www Package
  • hemlock_web Module