Job Details

INFRASTRUCTURE ENGINEER

Location: Oakland
Job ID: 15424

Job Posting

For UCOP internal applicants, please login to the internal candidate gateway at: Jobs at UCOP

UC OFFICE OF THE PRESIDENT

At the University of California (UC), your contributions make a difference. A world leader producing Nobel and Pulitzer prize recipients with over 150 years of groundbreaking research transforming the world. Choose a career where you can leverage your knowledge, skills and aspirations to inspire and support some of the greatest minds in the world, and those who will follow in their footsteps. Working at the University of California is being part of a unique institution, and a vibrant and diverse community. At the University of California, Office of the President, we propel our mission through impactful work locally, in government centers and systemwide. We are passionate people, serving the greater good.

The University of California, one of the largest and most acclaimed institutions of higher learning in the world, is dedicated to excellence in teaching, research and public service. The University of California Office of the President is the headquarters to the 10 campuses, five medical centers and three national laboratories and enrolls premier students from California, the nation and the world. Learn more about the UC Office of the President

DEPARTMENT OVERVIEW

The California Digital Library is a unit within the department of Academic Planning, Programs, and Coordination at the UC Office of the President. Gnter Waibel is the Associate Vice Provost and Executive Director, responsible for the direction, development, and management of the CDL.

The organizational structure of the CDL is based on four core programs, which are each led by a program manager and supported by a dedicated team of analysts and programmers. Additionally, the CDL has four groups that provide services to support the programs and the overall management of the broader organization.

POSITION SUMMARY

The Infrastructure & Applications Support (IAS) Infrastructure Engineer will help to create and support the technology infrastructure underlying all services provided by the California Digital Library (CDL), an administrative unit of the University of California Office of the President (UCOP). CDL provides world-class services in partnership with the ten UC campus libraries and the international scholarly community. These services have 4 main areas of focus: Publishing/Archives/Digitization, Discovery & Delivery, Shared Collections, and the UC Curation Center.

The IAS team is responsible for maintaining the technology framework upon which all CDL services are built, providing exceptional support to the CDL development teams as they deliver digital library solutions to the UC system, state of California and the world. The CDL infrastructure resides wholly within the AWS cloud and consists of over 200 EC2 instances, 60 RDS MySQL databases, hundreds of TBs of storage, and many other resources created from AWS service offerings. Our mission is to provide a stable, maintainable, recoverable and secure computing and network environment to enable and enhance existing and future operations while providing excellent technical support to the organization.

The IAS Infrastructure Engineer will be involved in operational support and system-level administration of CDL's AWS cloud infrastructure. Working closely with technical staff from all of the CDL program areas, they will create and configure the resources needed to build and deliver services in alignment with the CDL vision and mission. They will also coordinate and collaborate with central UCOP Information Technology Services (ITS) for project/task planning and review where there is an integration or overlap with the general UCOP infrastructure.

The successful candidate: Applies advanced systems infrastructure concepts and organizational objectives to resolve highly complex issues where analysis of situations or data requires an in-depth evaluation of variable factors. Selects methods, techniques and evaluation criteria to obtain results. Gives presentations to associated team and other technical units. Evaluates new technologies including performing moderate to complex cost/benefit analyses. Interacts with multiple levels of management. May lead a team of systems/infrastructure professionals, where appropriate. Performs advanced system administration and troubleshooting tasks, as an individual and part of a team. Delegates and mentors as appropriate. Maintains a good understanding of current trends and best practices within the information technology arena. Acts as a technical resource to other staff members and provides excellent technical support to the organization. Provides 24x7 on-call support as needed.

CDL welcomes and encourages diversity and seeks applications and nominations from individuals from underrepresented groups. CDL seeks to recruit and retain a diverse workforce as a reflection of our commitment to serve the people of California, to maintain the excellence of the University of California, and to offer our faculty and students richly varied disciplines, perspectives, and ways of knowing and learning.

Responsibilities

60%

UNIX/Linux System & Network Administration: Performs all complex technical support tasks necessary to ensure the nominal operation of the CDL technical infrastructure, including system design, configuration, implementation, maintenance, and modification, either independently, as a team member, or as a technical lead. Specifically, the position: Administers and supports AWS Linux instances running in the cloud. Creates new systems, updates existing systems, and troubleshoots and resolves all issues that are identified through monitoring tools or end user reports. Implements and maintains infrastructure applications including Nagios, Tripwire, Puppet, and LDAP. Evaluates, selects, and implements management utilities to increase the efficiency of the system administration staff and improve reporting capability regarding infrastructure status. Creates advanced scripts to automate routine system administration tasks and provide consistent results. Creates and implements Puppet manifests to manage common system configuration attributes. Monitors a support request queue, providing initial triage, consultation, and problem remediation. Reviews system/application logs on a regular basis, with responsibility to follow up and resolve any issues that are found. Plans, coordinates, and completes all required assignments on time and with high quality. Provides technical assistance and experience-based recommendations on a wide-range of complex technical issues to team members and the organization as a whole. Participates in organizational initiatives, including infrastructure design modifications, and provides recommendations for updates or changes to policies and standard operating procedures governing the technical infrastructure. Provides key input into departmental annual plan and individual project plans, establishes achievable task/project milestones and is responsible for accomplishing them. Sets daily and long-term priorities, both on an individual and on a team level (as appropriate). Coordinates project efforts between team, department, and organizational members. Creates and maintains documentation of processes, procedures, configurations, and standard tasks. Writes new documents to accompany new processes, procedures and tasks and updates existing documents to reflect changes in the infrastructure or operations. Reviews all documentation on a regular basis to ensure accuracy and usefulness. Works directly with web application teams to support their development efforts as required, including installing and configuring software (Apache, Tomcat, XTF, application frameworks, and tools). Provides assistance with functional and performance testing of applications, reviews architecture to ensure compliance with standard practices, and provides input into methods for continuous deployment and integration.

25%

AWS Administration: Designs, implements, and maintains complex cloud infrastructure components and services according to AWS standards and best-practices. Works with software development teams to specify and configure AWS services to support application requirements and audits the cloud environment to ensure consistent configurations & adherence to established standards. Independently manages AWS systems and services to support system-wide services provided by CDL operations. Performs complex analysis of infrastructure architecture and provides detailed recommendations for both immediate modifications and long-term improvements. Specifies, creates, and executes complicated scripts using the AWS command line interface to support system management, log analysis, and other maintenance activities on multiple, highly integrated systems. Implements and maintains complex security controls within the AWS infrastructure to enforce appropriate access and use of system and data. Creates and executes appropriate process and procedure necessary to ensure high-availability of all CDL services hosted within this environment. Evaluates and implements new AWS services as appropriate.

10%

Data & Network Security Administration: Creates and maintains a data protection program for a complex environment, using industry-standard best practices, to ensure the integrity and availability of all data within the infrastructure. This includes creating and managing backup and recovery mechanisms, implementing and enforcing security policies, and providing audit reporting used to assess risk associated with activity in the environment. Responsible for regular patching of systems and applications and reacting to specific security threats in accordance with established security policies. Provides recommendations and requirements for network, system, and data access controls. Manages user and role account permissions. Conducts periodic audit of established security controls and remediates any noted problems.

5%

Documentation: Creates and maintains documentation of processes, procedures, configurations, and standard tasks. Writes new documents to accompany new processes, procedures and tasks and updates existing documents to reflect changes in the infrastructure or operations. Reviews all documentation on a regular basis to ensure accuracy and usefulness.


Required Qualifications

• Demonstrated experience and skills in diagnosing and resolving application-level, operating system, database, storage, and networking problems.
• Demonstrated experience and skills in designing, implementing, and operating high-performance, high-availability system and storage architectures for mission critical online services.
• 5 to 7 years of experience working as a UNIX/Linux system administrator (or equivalent experience), including time spent fulfilling senior responsibilities.
• AWS certification (or equivalent) and at least 2 years' experience working in the Amazon AWS environment, including experience with AWS Linux system administration and management of AWS infrastructure services (EC2, EBS, S3). Experience with VPC, ELB, RDS and experience scripting with the AWS CLI a plus.
• Self-motivated and works independently and as part of a team. Able to learn effectively and meet deadlines Ability to work on and triage multiple high-priority tasks. Demonstrated complex problem-solving skills.
• Demonstrated effective communication and interpersonal skills. Demonstrated ability to communicate technical information to technical and non-technical personnel at various levels in the organization. Demonstrated excellence in written and oral communication. Demonstrated ability to interface with management on a regular basis.
• Understanding of change management best-practices and the importance of change control in maintaining highly-available systems and services. Experience using a version control system.
• Experience with key infrastructural technologies, including AWS EC2 and RDS/MySQL, Amazon Linux, Shibboleth, LDAP, Nagios, and Bash shell scripting. Experience writing and editing complex scripts used to perform system maintenance and administration.
• Demonstrated skills associated with adapting equipment and technology to serve the strategic and tactical goals of the organization. Comprehensive understanding of how system management actions affect other systems, system users and dependent/related functions.
• Demonstrated advanced knowledge, skills and abilities associated with system problem identification and resolution. Experience with design, configuration, operation, repair, and tuning of technology systems.
• Advanced knowledge of computer security best practices and policies including demonstrated experience securing most complex server-based software.

Preferred Qualifications

• Experience creating and maintaining an OpenLDAP authentication environment. Knowledge of Shibboleth a plus.
• Experience with routine database administration tasks in a MySQL environment.
• Cisco CCNA certification or equivalent experience creating and maintaining a complex network environment, including routing, subnetting, firewall, NACLs, and other standard components.
• Experience installing, configuring and maintaining the following tools: Tripwire, Nagios, Splunk and Puppet Enterprise.
• Experience as a technical lead; directing daily team operations, creating strategic plans and reporting to senior leadership.

Education

• Bachelor's degree in related area and/or equivalent experience/training. 5+ years' experience in software development or system administration required.

SALARY AND BENEFITS

Salary: $114K - $124K

Benefits: For information on the comprehensive benefits package offered by the University visit: Benefits of Belonging

ADDITIONAL INFORMATION

HOW TO APPLY

Please be prepared to attach a cover letter and resume with your application.

APPLICATION REVIEW DATE

The first review date for this job is March 5, 2021. The position will be open until filled.

CONDITIONS OF EMPLOYMENT

Background Check Process: Successful completion of a background check is required for this critical position. Background check process at UCOP

Smoke Free Work Environment: The University of California, Office of the President, is smoke & tobacco-free as of January 1, 2014. UC Smoke & Tobacco Free Policy

EEO STATEMENT

The University of California, Office of the President, is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, age or protected veteran status. EEO/AA Employer UC Nondiscrimination Policy

The University of California, Office of the President, strives to make this job board accessible to any and all users. If you have comments regarding the accessibility of our website or need assistance completing the application process, please contact us at: Accessibility or email the Human Resource Department at: epost@ucop.edu.

*LI-SG1



To apply, visit https://careerspub.universityofcalifornia.edu/psp/ucop/EMPLOYEE/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB.GBL?Page=HRS_APP_JBPST&Action=U&FOCUS=Applicant&SiteId=19&JobOpeningId=15424&PostingSeq=1

jeid-e04fd84eff004e4983321ab619aab64c