Resume: Thomas L. Kula
kula@tproa.net
kula.tproa.net/resume/
PGP Key Fingerprint: 9A43 21E9 8276 CC56 6B15
0136 5E07 A06E 81C0 4D0E
Interest: Quickly and deeply understanding complex technologies to
help others understand and adopt pragmatic solutions to
challenging problems
Work Experience
2024 - Now: Sr. Staff Solutions Architecture Specialist
2022 - 2024: Sr. Staff Solutions Engineer
2019 - 2022: Staff Solutions Engineer
2018 - 2019: Sr. Solutions Engineer
HashiCorp
Solutions engineer on an internal team building material
to support field solutions engineers across our organization.
Demonstrated a strong ability to communicate the value of our
enterprise software to developers, networking, operations, and
security teams as well as all levels of management and
leadership. Familiar with related technologies and how they
interoperate to provide a complete solution to customers.
Responsible for developing training for internal staff,
leading multiple workshops both internally and for
customers/opportunities from small groups into marketing
workshops numbering in the hundreds. Worked with several
large customers across a variety of verticals to develop
PoC plans for their internal testing, setting a standard
for PoC plan comprehensiveness.
Independently discovered and successfully advocated for the
publication and fixing of CVE-2021-32574. Discovered
HCSEC-2022-20/CVE-2022-40716.
Familiar with all of the HashiCorp suite and how they
inter-operate, primarily focused on Consul, with another key
strength being HashiCorp Vault. When serving as a field
solutions engineer I led several large workshops, both in
person and online, public and for customers, covering Consul,
Vault and Terraform. Assisted in authoring major cloud
assessments and serivce mesh architectural documents for a
Fortune 50 company. Experienced in all phases of the sales
journey, from discovery, to technical validation, proof-of-
value and proof-of-concept execution, and setting up post-sales
success and eventual expansion.
Deep experience in the Instruqt learning platform, from
creating and maintaining tracks, managing the platform, and
automating workflows via the Instruqt GraphQL API. Deep
experience with GitHub actions.
* Emerging Products SE of the Year 2022
* HashiCorp Sales Club 2022
* Americas SE of the Quarter 2020Q2
* North America - NY Metro MVP 2020Q1
* Americas Enterprise East SE of the Quarter 2020Q1
* Americas SE of the Quarter 2020Q1
Major projects listed below.
2014 - 2018: Senior Systems Engineer
Systems Engineer
Technical Operations Team
Birchbox, Inc
Work on a small team providing core infrastructure and
office IT support for the company. Technologies include
Debian Linux, Docker, Mesos/Marathon, Varnish, Apache,
Nginx, Haproxy, Consul, Java JRE, memcached, RabbitMQ and
others. Moving legacy Chef infrastructure to Salt Stack.
Providing storage using FreeNAS, identity management with
Kerberos/OpenLDAP. Experience managing Juniper network
equipment/JunOS, bare metal switches with Cumulus Linux,
and Meru wireless controllers. Experience in automating
workflows in AWS, with particular focus on IAM management
and security. Acting as technical lead for our ongoing PCI
Compliance efforts.
Major projects listed below.
2011 - 2014: Senior Systems Engineer
Libraries Information Technology Office, The Libraries
Columbia University in the City of New York
Part of a small team responsible for Linux systems
providing core infrastructure and web hosting for various
Libraries sites and services. Management of CentOS systems
primarily as Xen hosted VMs, using Salt Stack, maintaining
LAMP, Tomcat and Rails infrastructure, monitoring using
Graphite. Developing scripts for systems management and
monitoring, user administration and general system
administration, primarily in Python. Wrote a Python module
to aid in managing users and groups on Atlassian Confluence.
Developed and wrote a more robust MySQL backup system.
Redesigned and rewrote a script to synchronize local files
to HSI storage provided by a partner school. Experience in
web authentication systems, kerberization of services,
building systems with ZeroMQ, and packaging software for use
with RPM/Yum. Worked with central IT staff to provision
storage (SAN and NAS) and to interface with various central
services, including LDAP and legacy user management systems.
Started as Systems Engineer, was promoted to Senior Systems
Engineer in 2013.
Major projects are described below.
2010 - 2011: Client/Server Programmer Senior
Information and Technology Services
University of Michigan, Ann Arbor, MI
An extension of the next listing, with a different job
title (following a reorganization) and at the senior
level.
2008 - 2010: Software Engineer Intermediate
Information and Technology Services
University of Michigan, Ann Arbor, MI
An extension of the next listing, with more development
responsibilities. Working on several mid-sized projects
primarily in Perl and C, working on a team responsible for
interfacing a large identity management system with the
services our group provides. Major work on provisioning
automation, monitoring and robustness testing of the
department CIFS offering. An emphasis on developing systems
that are robust, scalable and secure, and on providing
services that delegate administrative authority to select
groups of people with corresponding vetting and auditing.
Areas of focus include functional and load testing of
systems. Performed Kerberos database password audits,
ongoing involvement in TSM operation, planning and design.
Major projects are described in detail below.
2006 - 2008: System Administrator Intermediate
Information and Technology Services
University of Michigan, Ann Arbor, MI
Served on the group responsible for campus-wide Kerberos
and AFS file service, general-purpose and statistics
Unix computing services. Responsibilities required being
able to understand and use these technologies at an
intermediate to high level of understanding, work
independently and with co-workers, campus IT providers
and end-users to diagnose and solve problems as well
as helping others use the services properly. Extensive
experience administering systems using Radmind. Also
served as part of the group responsible for U-M
hostmaster services using ISC bind and dhcpd, requiring
a solid understanding of DNS and being able to help campus
IT providers as well as non-savvy end users utilize that
service.
Implemented the new campus TSM service and served as
part of the team moving clients from the old AIX-based
TSM service to the new service (see Projects below).
2006 - 2006: Systems Administrator
Information Technology Services
Iowa State University, Ames, IA
Split responsibilities between OS X lab deployment and
AFS/backup administration.
Designed and implemented OS X lab deployment system
using NetRestore and custom installation/configuration
scripts.
Assist in maintaining Teradactyl TiBS backup system,
providing backup services for the ISU AFS cell and
various other central servers. Testing new AFS file servers
and clients. Other assorted Unix administration tasks,
primarily Red Hat Enterprise Linux and NetBSD.
Continuing duties in print queue creation, greylisting
and e-mail problems, short course development, Linux
lab development.
2001 - 2006: System Support Specialist
Information Technology Services
Iowa State University, Ames, IA
Provided technical support as part of Iowa State's central IT
help desk. Specialized in OS X and Unix support, supporting
central Kerberos and AFS services, and VPN support. Primary
contact for creation of central print queues. First contact
for e-mail greylisting problems. Responsible for creating
documentation and FAQs for end-user support, and developing
parts of a series of short courses on Unix use and system
administration. Miscellaneous other training.
Projects include Linux Localization and developing a general
purpose Linux lab.
Leadership
2011 - 2015: Trevorspace/Ask Trevor/Trevor NextGen NYC Volunteer
The Trevor Project, New York City
Volunteer in various capacities with the Trevor Project, which
provides suicide prevention and crisis intervention services
to lesbian, gay, bisexual, transgender and questioning youth
to 24 years of age. Trained as a Ask Trevor author, answering
letters submitted by youth to the Ask Trevor web site and as
a Trevorspace administrator, monitoring Trevorspace, the Trevor
Project's safe social networking site for youth. Also work with
Trevor NextGen NYC, a group of young volunteers in New York
City that does projects, programming, community outreach and
fund raising for the Trevor Project.
2010 - 2011: Member, Non-Motorized Transportation Advisory Committee
City of Ypsilanti, Michigan
A committee of the city Planning Commission, serving to help
implement the city non-motorized transportation plan through
research and recommendations to the Planning Commission.
2009 - 2011: Member, Board of Directors
Ypsilanti Food Cooperative
Along with the other directors work closely with the co-op
general manger to represent the member-owners of the cooperative,
provide oversight, strategic planning and long-term goal
making for the cooperative. Developed a basic understanding
of financial statements and local, state and federal laws
affecting both a cooperative and a grocery store.
Education
2000: Drake University, Des Moines, IA
Bachelor of Science
Majors: Computer Science and Mathematics
Certifications
2020: Vault Associate Exam Contributor
HashiCorp, Inc
Issued: 15 April 2020
Expires: 15 April 2022
https://www.youracclaim.com/badges/05634cec-d942-4278-9d43-ed214a5f0f09
Earners of the HashiCorp Certified: Vault Associate Exam Contributor
certification have contributed significantly to building the HashiCorp
Vault Associate exam. These subject matter experts are the cloud
engineers who helped build and maintain this exam.
This person has completed one or more of the following: 1) Has written
at least 10 accepted questions, 2) Has reviewed at least 20 questions,
and 3) Was a significant contributor to the exam role scope
Major Projects:
2023: CircleCI to GitHub Actions Migration
As part of an organization-wide move to GitHub Actions
migrated nearly 30 repositories for the Solutions Engineering
team, starting with knowing essentially nothing about GHA.
Migrated repositories with multiple inter-related jobs and
dependencies, integrated with HashiCorp Vault and Slack
notifications.
Examples of migrations can be found at:
https://github.com/hashicorp/field-workshops-consul/pull/206
https://github.com/hashicorp/field-workshops-vault/pull/95
https://github.com/hashicorp/field-workshops-terraform/pull/362
https://github.com/hashicorp/field-workshops-nomad/pull/147
2016 - 2018: PCI Compliance Efforts
Acting as technical lead for all of our PCI DSS compliance
efforts, working closely with a project manager and all
business units to maintain and move to a higher level of
PCI compliance. Working familiarity with PCI DSS 3.2, working
with ASVs and QSAs. Primarily responsible for all PCI-mandated
documentation, and driving the design process for isolating
production payments systems and our cardholder-data environment.
Acting as local technical expert on PCI DSS requirements and
how they affect all business units. Deep experience in particular
on designing and changing the technical infrastructure and
business operations of a maturing startup business.
2017: London Office Buildout
Coordinated the IT aspects of our London office moving locations.
Worked closely with buildout contractor to design and specify
layout of "comms closet" and ethernet drops throughout office.
Picked network gateway and switching equipment, designed office
network, and worked closely with contractor, building facilities
and network providers to install fibre network service.
Implemented backup ad-hoc solution to overcome delay in incumbent
fibre provider delivery date. On-site for six days to physically
install all networking equipment, implementing last-minute
solutions to problems inherent in all buildouts. Passing
familiarity with UK telecommunications providers and wayleave
process.
2015 - 2016: Internal File Services
Worked as part of a team migrating company Dropbox usage to
either Google Docs or, for specialized needs, a FreeNAS
cluster. Engineered FreeNAS system, including integration
with corporate OpenLDAP directory for CIFS login. Responsible
for hardware selection and engineering snapshot policies and
replication between sites.
2013 - 2014: Ad-hoc storage service
Developing a mid-tier 'ad-hoc' storage service to provide a
level of network storage above 'disks thrown in random
machines' but not requiring expensive preservation storage.
Engineering a solution using two systems running FreeNAS
replicated across campus.
2013 - 2014: Implementation and Migration to Isilon Storage Cluster
Working with an outside vendor, planned and implemented
the installation of a two-site Isilon storage cluster.
Worked with data center and networking staff for appropriate
resources, researched and created appropriate replication
policies, and implemented monitoring and statistics gathering.
Worked with other groups within the libraries to plan the
migration and cutover of their data from an older storage
system to this cluster. Developed process to sync contents
to a third site, using HSI storage obtained under a
partnership with Indiana University.
2012 - 2014: Implementation of Monitoring using Graphite
Installed the Graphite monitoring tool, including
customization to protect the web front end behind a local
web authentication system. Developed or modified several
already existing scripts to stream various system and
application metrics into Graphite. Wrote a Graphite relay
that publishes metrics on a ZeroMQ pub socket, and an
application that allows for watching metrics based on a
pattern. Wrote software to perform alerting based on
metrics published in Graphite.
2010 - 2011: Mainstream Storage
Involved in several aspects of the ITS CIFS storage offering,
called "Mainstream Storage", using IBM re-branded NetApp
gateways backed by an IBM SVC SAN. Developed and executed
comprehensive test plan to exercise and verify correctness of
gateway cluster failover. Use the NetApp native API, ONTAPI,
via Perl to work on automation of storage provisioning to
clients. Patching ONTAPI perl interface to allow for gateway
SSL certificate verification before sending administrative
credentials. Development of Powershell over SSH to provision
Windows-specific parts of the storage. Worked with other TSM
team members to test use of NetApp/TSM snapdiff backups, and
develop workarounds when that does not work properly.
http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/storage/mainstream/
2009 - 2010: Kerberos and AFS identity provisioning
Developed and tested software used by our Identity Management
system to provision Kerberos principals and AFS protection
identities. Based on remctl and written in Perl, replacing
similar software that was written in Java. As part of development
wrote extensive unit tests. As a security sensitive service,
focused on correctness and robustness, as a critical part of
account provisioning focused on reliability.
http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/iaa/jkservices/
2009: Kerberos Password Quality Plugin
Using a locally modified plugin architecture for MIT Kerberos 1.6.2,
wrote, tested and deployed a plugin providing password quality
checking written in C and utilizing the cracklib library. As a plugin
embedded in the Kerberos administrative server, focused major effort
on ensuring security and robustness. Extensive testing with valgrind
to identify and eliminate resource exhaustion. Identified file
descriptor leaking in the cracklib code, developed deployment changes
to keep that from causing problems, and identified future code
changes to eliminate the problem.
As an auxiliary project, worked with our web development team to
build a remctl-based backend that allowed the next generation of
the web password change page to provide near instant feedback
to users of the quality of a password as it is being typed.
http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/iaa/kadmind-pw-strength-plugin/
2009: Group Home Directory Automation
Developed a system to replace a highly-inefficient manual process
for provisioning AFS group home directories with an automated system
which allows end users to directly create them via a web interface.
Primarily responsible for the backend of the system, working with
the web development team to define a remctl-based interface used
by the web frontend.
During testing phase, identified an as of yet unresolved issue
either in the Perl AFS modules or the AFS Rx package causing
spurious crashes. Developed work-around requiring re-design
of the software to isolate various parts of the backend in
independent sub-processes.
http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/ifs/homedir/
2008 - 2009: Delegated User Disable Service
Developed system to allow for User Advocate and Security
Services staff to have limited delegated access to disable
user Kerberos principals and to lock out access to user
AFS home directories. Extensive focus on auditing, both from
a security standpoint and for allowing support staff to identify
disabled users and who was responsible for the disabling.
http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/ifs/disableuser/
2007 - 2008: U-M ITS TSM service
Implementation of the new ITS Tivoli Storage Manager backup
service. Took specifications from outside consultant and
developed and implemented the new service, moving it from
a monolithic AIX-based service to a modular Linux-based
service. Developed loadset for the TSM servers, working to
implement kernel requirements, driver requirements for
fibre channel cards and adapting TSM software to work and
be managed as part of a UMCE Linux distribution. Wrote
the bulk of the scripts used to manage the new system,
with the goal of allowing a modular and distributed
system, with many more machines than in the previous
service, to be managed effectively and allowing new
resources to be slotted in easily without requiring
major changes in management. Worked as part of the team
maintaining the legacy and new TSM service and moving
clients to the new service. Continued involvement in
architecture and capacity planning.
Presentations
2023: "Why You Should Use Vault as your Consul CA"
Presented at HashiTalks 2023
https://kula.tproa.net/lnt/2023/02/why-you-should-use-vault-as-your-consul-certificate-authority/
2015: "Ship It! Containerizing your KDCs"
Presented at the 2015 AFS and Kerberos Best Practices
Workshop, Pittsburgh, PA
https://kula.tproa.net/talks/afskbpw2015/afskbpw2015-kula.pdf
A brief overview of containers, and how they can be used
to host kerberos KDCs.
2010: "Managing Suck: Kerberos Password Quality at the University of
Michigan"
Presented at the 2010 AFS and Kerberos Best Practices
determining, programmatically, what exactly is a bad password,
as well as how password quality should fit into a broader
security strategy.
2009: "Hacking AFS Dumps for Fun and Profit"
Presented at the 2009 AFS and Kerberos Best Practices
Workshop, Stanford University
https://kula.tproa.net/talks/afskbpw2009/kula-afs-dumps-2009.pdf
Using the information present in AFS volume dumps for useful
purposes, and presenting rough code to use this, as well as an
outline for future development.
2008: "Introducing pyremctl, and a case study in using remctl"
Presented at the 2008 AFS and Kerberos Best Practices
Workshop, New Jersey Institute of Technology
http://workshop.openafs.org/afsbpw08/talks/thu_1/kula-pyremctl.pdf
http://workshop.openafs.org/afsbpw08/talks/thu_1/kula-umich-remctl.pdf
A brief overview of the Python remctl client library bindings, and
some case studies in the use of remctl at the University of Michigan.
2007: "Xen as a Test Environment"
Presented at the 2007 AFS and Kerberos Best Practices
Workshop, Stanford University
https://kula.tproa.net/talks/afsbpw2007/afsbpw2007-kula.pdf
Using Xen para-virtualization as a test environment for
Kerberos and AFS services.
2006: "iRealm: Explorations in using OS X to provide AFS and Kerberos
Services"
Presented at the 2006 AFS and Kerberos Best Practices
Workshop, University of Michigan
https://kula.tproa.net/talks/afsbpw2006
Presentation on using OS X Server to provide AFS and Kerberos
services. Step-by-step instructions on deploying AFS services
on OS X Server, discussion of caveats and practicality of
using OS X server to provide these services.
2005: "NetBSD, AFS and Kerberos: From Zero to Distributed File
System in N Easy Steps"
Co-authored with Tracy Di Marco White, Iowa State University
Presented at the 2005 AFS and Kerberos Best Practices
Workshop, Carnegie Mellon University
https://kula.tproa.net/talks/afsbpw2005
Step-by-step instructions on providing AFS and Kerberos services
using NetBSD, Heimdal, OpenAFS and Arla.
Contributions to open-source software:
- https://github.com/kula/
https://github.com/thomashashi/
- HashiCorp Vault: Bulk of work to add 'service' and 'node' identity
capabilities to the Consul secret engine
https://github.com/hashicorp/vault/pull/15295
- HashiCorp Terraform: Update 'aws_route' to handle changes in IPv6 route entries
https://github.com/terraform-providers/terraform-provider-aws/pull/12062
- HashiCorp Vault: Allow auto_auth with templates without specifying a sink
https://github.com/hashicorp/vault/pull/8812
- A secrets engine plugin for HashiCorp Vault to provision users
in the Minio objects storage server.
<https://github.com/kula/vault-plugin-secrets-minio>
- A secrets engine plugin for HashiCorp Vault to provision application
keys for the Backblaze B2 object storage service.
<https://github.com/kula/vault-plugin-secrets-backblazeb2>
- Update minio documentation to reflect changes in ARN default
region handling.
<https://github.com/minio/minio/pull/5101>
- Initial development and ongoing maintenance of Python
remctl client bindings, included as part of the stock
remctl distribution from 2.13 forward.
<http://www.eyrie.org/~eagle/software/remctl/python-readme.html>
- Developed Go bindings for the remctl client libaries
<https://github.com/kula/go-remctl>
- Provide minor bug-fixes and testing of k5start
<http://www.eyrie.org/~eagle/software/kstart/> and remctl
<http://www.eyrie.org/~eagle/software/remctl/>, primarily
under NetBSD.
- Added support to remctld to set the environment variable
REMCTL_COMMAND, present in remctl 2.16 and on
- Patch to pam-afs-session not to delete afs credentials
if DELETE_CRED is called but retain_after_close is configured.
<http://www.eyrie.org/~eagle/software/pam-afs-session/>
- Submitted patch to enhance Heimdal Kerberos support in
the FreeRadius rlm_krb5 module, improving logging of
authentications with Kerberos principals that have
instances and allowing the module to use a non-default
keytab and service principal.
<http://lists.freeradius.org/mailman/htdig/freeradius-devel/2007-April/011021.html>
- Initial rough implementation of "server"-side zephyr
braindump authentication in the Kerberos 5 variant of the
Zephyr messaging service.
<http://kula.tproa.net/code/k5zephyr-bdumps-tproa.diff>
- Patches to NetBSD pkgsrc OpenAFS and Arla packages to allow
more peaceful co-existance:
<http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=33399>
- Small unsubmitted patch to Pubcookie that removes apache
installation path assumptions:
<http://kula.tproa.net/stuff/pubcookie-3.3.2d-tproa.diff>
- Patch OpenAFS vol-dump to handle >2GB dump files and do
incremental dumps:
DELTA vol-dump-incr-largefile-support-20081222
<http://www.openafs.org/cgi-bin/wdelta/MAIN/vol-dump-incr-largefile-support-20081222>
<http://rt.central.org/rt/Ticket/Display.html?id=123984>
- Patch to add GZIP file support to the CMU SCS xfile package,
and bugfix to the dumpscan library:
<http://kula.tproa.net/code/xfile-gzip.patch>
<http://kula.tproa.net/code/dumpscan-dont-call-cb_dirent-twice.patch>
- OpenAFS change I4f9bcbae: Add -usetokens option to libadmin test
'afscp'
<http://gerrit.openafs.org/#change,3899>
- Patch the Python getent module so group lookups work properly
https://github.com/tehmaze/getent/pull/1#issuecomment-8359720
- Allow for minion reconnect backoff in Salt Stack
https://github.com/saltstack/salt/pull/6360
- Fix HTTP authentication support in the Salt Stack cp module
https://github.com/saltstack/salt/pull/6356
- Write documentation for the Salt Stack ext_pillar facility
https://github.com/saltstack/salt/pull/4318
Other experience not mentioned elsewhere
Comfortable with Bash, Python, and Golang. Can handle enough C to get along.
$Date: 2024-02-05T16:22:31-05:00$
$Version: bfe19b2ae781e97990e750d0754c3d41d2e7a7fb$