Resume: Thomas L. Kula kula@tproa.net Interest: Building large-scale, robust and secure distributed systems Work Experience 2008 - Now: Software Engineer Intermediate Information and Technology Services University of Michigan, Ann Arbor, MI An extension of the next listing, with more development responsibilities. Working on several mid-sized projects primarily in Perl and C, working on a team responsible for interfacing a large identity management system with the services our group provides. Major work on provisioning automation, monitoring and robustness testing of the ITS CIFS offering. An emphasis on developing systems that are robust, scalable and secure, and on providing systems that delegate administrative authority to select groups of people with corresponding vetting and auditing. Areas of focus include functional and load testing of systems. Performed Kerberos database password audits, ongoing involvement in TSM operation, planning and design. Major projects are described in detail below. 2006 - 2008: System Administrator Intermediate Information and Technology Services University of Michigan, Ann Arbor, MI Served on the group responsible for campus-wide Kerberos and AFS file service, general-purpose and statistics Unix computing services. Responsibilities required being able to understand and use these technologies at an intermediate to high level of understanding, work independently and with co-workers, campus IT providers and end-users to diagnose and solve problems as well as helping others use the services properly. Extensive experience administering systems using Radmind. Also served as part of the group responsible for U-M hostmaster services, requiring a solid understanding of DNS and being able to help campus IT providers as well as non-savvy end users utilize that service. Implemented the new campus TSM service and served as part of the team moving clients from the old AIX-based TSM service to the new service (see Projects below). 2006 - 2006: Systems Administrator Information Technology Services Iowa State University, Ames, IA Split responsibilities between OS X lab deployment and AFS/backup administration. Designed and implemented OS X lab deployment system using NetRestore and custom installation/configuration scripts. Assist in maintaining Teradactyl TiBS backup system, providing backup services for the ISU AFS cell and various other central servers. Testing new AFS file servers and clients. Other assorted Unix administration tasks, primarily Red Hat Enterprise Linux and NetBSD. Continuing duties in print queue creation, greylisting and e-mail problems, short course development, Linux lab development. 2001 - 2006: System Support Specialist Information Technology Services Iowa State University, Ames, IA Provided technical support as part of Iowa State's central IT help desk. Specialized in OS X and Unix support, supporting central Kerberos and AFS services, and VPN support. Primary contact for creation of central print queues. First contact for e-mail greylisting problems. Responsible for creating documentation and FAQs for end-user support, and developing parts of a series of short courses on Unix use and system administration. Miscellaneous other training. Projects include Linux Localization and developing a general purpose Linux lab. Leadership 2009 - Now: Member, Board of Directors Ypsilanti Food Cooperative Along with the other directors work closely with the co-op general manger to represent the member-owners of the cooperative, provide oversite and strategic planning and long-term goal making for the cooperative. Developed a basic understanding of financial statements and local, state and federal laws affecting both a cooperative and a grocery store. Education 2000: Drake University, Des Moines, IA Bachelor of Science Majors: Computer Science and Mathematics Major Projects: 2010: Mainstream Storage Involved in several aspects of the ITS CIFS storage offering, called "Mainstream Storage", using IBM rebranded NetApp gateways backed by an IBM SVC SAN. Developed and executed comprehensive test plan to exercise and verify correctness of gateway cluster failover. Use the NetApp native API, ONTAPI, via Perl to work on automation of storage provisioning to clients. Patching ONTAPI perl interface to allow for gateway SSL certificate verification before sending administrative credentials. Development of Powershell over SSH to provision Windows-specific parts of the storage. Worked with other TSM team members to test use of NetApp/TSM snapdiff backups, and develop workarounds when that does not work properly. http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/storage/mainstream/ 2009 - 2010: Kerberos and AFS identity provisioning Developed and tested software used by our Identity Management system to provision Kerberos principals and AFS protection identities. Based on remctl and written in Perl, replacing similar software that was written in Java. As part of development wrote extensive unit tests. As a security sensitive service, focused on correctness and robustness, as a critical part of account provisioning focused on reliability. http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/iaa/jkservices/ 2009: Kerberos Password Quality Plugin Using a locally modified plugin architecture for MIT Kerberos 1.6.2, wrote, tested and deployed a plugin providing password quality checking written in C and utilizing the cracklib library. As a plugin embedded in the Kerberos administrative server, focused major effort on ensuring security and robustness. Extensive testing with valgrind to identify and eliminate resource exhaustion. Identified file descriptor leaking in the cracklib code, developed deployment changes to keep that from causing problems, and identified future code changes to eliminate the problem. As an auxiliary project, worked with our web development team to build a remctl-based backend that allowed the next generation of the web password change page to provide near instant feedback to users of the quality of a password as it is being typed. http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/iaa/kadmind-pw-strength-plugin/ 2009: Group Home Directory Automation Developed a system to replace a highly-inefficient manual process for provisioning AFS group home directories with an automated system which allows end users to directly create them via a web interface. Primarily responsible for the backend of the system, working with the web development team to define a remctl-based interface used by the web frontend. During testing phase, identified an as of yet unresolved issue either in the Perl AFS modules or the AFS Rx package causing spurious crashes. Developed work-around requiring re-design of the software to isolate various parts of the backend in independent sub-processes. http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/ifs/homedir/ 2008 - 2009: Delegated User Disable Service Developed system to allow for User Advocate and Security Services staff to have limited delegated access to disable user Kerberos principals and to lock out access to user AFS home directories. Extensive focus on auditing, both from a security standpoint and for allowing support staff to identify disabled users and who was responsible for the disabling. http://cvs.itd.umich.edu/cgi-bin/cvsweb.cgi/ifs/disableuser/ 2007 - 2008: U-M ITS TSM service Implementation of the new ITS Tivoli Storage Manager backup service. Took specifications from outside consultant and developed and implemented the new service, moving it from a monolithic AIX-based service to a modular Linux-based service. Developed loadset for the TSM servers, working to implement kernel requirements, driver requirements for fibre channel cards and adapting TSM software to work and be managed as part of a UMCE Linux distribution. Wrote the bulk of the scripts used to manage the new system, with the goal of allowing a modular and distributed system, with many more machines than in the previous service, to be managed effectively and allowing new resources to be slotted in easily without requiring major changes in management. Worked as part of the team maintaining the legacy and new TSM service and moving clients to the new service. Continued involvement in architecture and capacity planning. Presentations 2010: "Managing Suck: Kerberos Password Quality at the University of Michigan" Presented at the 2010 AFS and Kerberos Best Practices Workshop, University of Illinois, Urbana-Champaign http://kula.tproa.net/talks/afskbpw2010/kula-managing-suck.pdf A discussion of the development of a password quality plugin for the UMICH.EDU Kerberos realm and some of the resultant unforseen problems and lessons learned. Discussion of the utility of password quality plugins and the difficulty of determining, programmatically, what exactly is a bad password, as well as how password quality should fit into a broader security strategy. 2009: "Hacking AFS Dumps for Fun and Profit" Presented at the 2009 AFS and Kerberos Best Practices Workshop, Stanford University http://kula.tproa.net/talks/afskbpw2009/kula-afs-dumps-2009.pdf Using the information present in AFS volume dumps for useful purposes, and presenting rough code to use this, as well as an outline for future development. 2008: "Introducing pyremctl, and a case study in using remctl" Presented at the 2008 AFS and Kerberos Best Practices Workshop, New Jersey Institute of Technology http://workshop.openafs.org/afsbpw08/talks/thu_1/kula-pyremctl.pdf http://workshop.openafs.org/afsbpw08/talks/thu_1/kula-umich-remctl.pdf A brief overview of the Python remctl client library bindings, and some case studies in the use of remctl at the University of Michigan. 2007: "Xen as a Test Environment" Presented at the 2007 AFS and Kerberos Best Practices Workshop, Stanford University http://kula.tproa.net/talks/afsbpw2007/afsbpw2007-kula.pdf Using Xen para-virtualization as a test environment for Kerberos and AFS services. 2006: "iRealm: Explorations in using OS X to provide AFS and Kerberos Services" Presented at the 2006 AFS and Kerberos Best Practices Workshop, University of Michigan http://kula.tproa.net/talks/afsbpw2006 Presentation on using OS X Server to provide AFS and Kerberos services. Step-by-step instructions on deploying AFS services on OS X Server, discussion of caveats and practicality of using OS X server to provide these services. 2005: "NetBSD, AFS and Kerberos: From Zero to Distributed File System in N Easy Steps" Co-authored with Tracy Di Marco White, Iowa State University Presented at the 2005 AFS and Kerberos Best Practices Workshop, Carnegie Mellon University http://kula.tproa.net/talks/afsbpw2005 Step-by-step instructions on providing AFS and Kerberos services using NetBSD, Heimdal, OpenAFS and Arla. Contributions to open-source software: - Initial development and ongoing maintenance of Python remctl client bindings, included as part of the stock remctl distribution from 2.13 forward. - Provide minor bug-fixes and testing of k5start and remctl , primarily under NetBSD. - Added support to remctld to set the environment variable REMCTL_COMMAND, present in remctl 2.16 and on - Patch to pam-afs-session not to delete afs credentials if DELETE_CRED is called but retain_after_close is configured. - Submitted patch to enhance Heimdal Kerberos support in the FreeRadius rlm_krb5 module, improving logging of authentications with Kerberos principals that have instances and allowing the module to use a non-default keytab and service principal. - Initial rough implementation of "server"-side zephyr braindump authentication in the Kerberos 5 variant of the Zephyr messaging service. - Patches to NetBSD pkgsrc OpenAFS and Arla packages to allow more peaceful co-existance: - Small unsubmitted patch to Pubcookie that removes apache installation path assumptions: - Patch OpenAFS vol-dump to handle >2GB dump files and do incremental dumps: DELTA vol-dump-incr-largefile-support-20081222 - Patch to add GZIP file support to the CMU SCS xfile package, and bugfix to the dumpscan library: Other experience not mentioned elsewhere Maintain small network providing various services to about 25 users and volunteer groups, using NetBSD, Xen, Heimdal, Arla, OpenAFS, Cyrus imap, apache and postfix. Experimenting with cross-realm Kerberos, zephyr and AFS with Iowa State and other local groups. Use OpenVPN to participate in multi-site, multi- state virtual private network. Comfortable with sh, Python and Perl scripting, coding in C; familiar with PostgreSQL. $Date: 2010/08/18 22:14:25 $ $Id: resume.txt,v 1.4 2010/08/18 22:14:25 kula Exp $