Senior Data Engineer
Office for National Statistics
Apply before 11:55 pm on Tuesday 15th October 2024
Details
Reference number
Salary
Job grade
Contract type
Business area
Type of role
Information Technology
Working pattern
Number of jobs available
Contents
Location
ONS operates a flexible hybrid working model across the UK, with colleagues linked to one of our contractual locations in Newport, Titchfield (Fareham), Manchester, London or Edinburgh and working between office and remote throughout the week.
As part of the hybrid working arrangement there is a 40% minimum office attendance requirement. Attendance is typically at your contractual office, with occasional travel to alternative locations. Due to estates constraints, there are currently temporary exceptions to this for colleagues based at Manchester and London who are required to attend the office for a minimum of 20% of their work time.
About the job
Job summary
This is an exciting opportunity to join our Integrated Data Sharing platform (IDS) Data Engineering team. The IDS Programme aims to enable data sharing and innovative cross-cutting analysis across and at the heart of government through a single platform, and represents a large step-change in how government utilises data.
This role will be at the core of this, designing, maintaining existing and building new systems, pipelines and data products within a team of GCP Data Engineers.
As well as the day-to-day processing and delivery, we work as a team to develop the capability of the organisation – developing re-usable and increasingly automated code, tools, and best practice that we and others across the organisation can use to build the data pipelines needed to support the growing work of a changing organisation.
Job description
Our tech stack is 60% Python, PySpark and SQL and the remaining 40% GCP eco system tools. We’re looking for someone with relevant technical skills but also a great attitude, who is proactive, takes pride in the quality of their team’s work and has an eagerness to develop their own technical skills and their team’s.
Training will be provided in the aspects of the work the successful candidate is not familiar with and they will be buddied with an experienced peer. This role will give exposure to the GCP eco system, Pub/Sub, Cloud functions, AirFlow, GitHub actions, CI/CD, DevOps practices and Agile project delivery with cross-government stakeholders; all while working alongside other highly skilled GCP Data and Platform Engineers.
We encourage training and development so that we can stay on top of changing technology. We invest in the development of our staff with extensive internal and external training opportunities.
Responsibilities
Responsibilities include but are not limited to:
- Designing and building robust data pipelines, ETL processes, data validation and other products using Python, SQL, BigQuery, Spark, DataProc, AirFlow and wider GCP eco system. Maintenance of existing pipelines.
- Orchestrating services and systems using Pub/Sub and AirFlow.
- Manage our data estate.
- Investigation and mitigation of data quality issues by implementing appropriate solutions.
- Actively seek to improve existing pattern and best practices. Encourage and lead discussions of team standards and their implementation collaboratively.
- Producing a high standard of technical work and non-technical, setting an example for the team.
- Horizon scanning for opportunities and risks, planning future work and actions to mitigate risks.
- Proactively using self-development time to grow own technical and non-technical skillsets. Encourage initiatives that build a culture of self-development across the team.
- Becoming an expert in areas key to the project.
Person specification
Essential Skills Criteria:
- Technical Skills: Experience utilising Python, PySpark, SQL and GIT.
- Programming and build: You know how to use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations. You can collaborate with others to review specifications where appropriate.
- Data development process: You know how to establish enterprise-scale data integration procedures across the data development life cycle and ensure that teams adhere to these. You can manage resources to ensure that data services work effectively.
- Problem resolution: You can ensure that the most appropriate actions are taken to resolve problems as they occur. You know how to coordinates teams to resolve problems and implement solutions and preventative measures.
- Communication skills: You can listen to the needs of technical and business stakeholders and interpret them. You are flexible and capable of proactive and reactive communication. You know how to facilitate difficult discussions within the team or with diverse senior stakeholders.
Desirable Skills:
- Experience working on GCP
Behaviours
We'll assess you against these behaviours during the selection process:
- Making Effective Decisions
- Changing and Improving
- Working Together
Technical skills
We'll assess you against these technical skills during the selection process:
- Programming and build - Experience of designing and building data pipelines using Python and Spark eco systems following industry best practices.
- Problem resolution - Problem-solving skills and ability to troubleshoot data pipeline issues, analyse logs and error messages, and implement effective solutions to ensure continuous data flow.
Benefits
The Office for National Statistics is part of the Civil Service, and as such we share a number of key benefits with other departments, whilst also having our own unique offerings to support our 5400 valued colleagues across the business.
This role is part of the cross-government Government Digital and Data (formerly DDaT) profession framework. As a role within Government Digital and Data (formerly DDaT) at the ONS, we also offer benefits such as:
• Protected Learning Time to spend on your personal development and side-projects.
• A supportive and active Community of Practice which you will be expected to contribute to, helping ensure you and your colleagues get the training, development and opportunities you need to progress your careers.
ONS are committed to flexible ways of working that support a healthy work-life balance. ONS has already considered how this job could be right sized for applicants working flexibly and we are happy to explore options with you about working part time, in a job share or flexibly, in line with our hybrid working policies.
Whether you are hearing about us for the first time or already know a bit about our organisation, we hope that the benefits pack attached (bottom of page) will give you a great insight into the benefits and facilities available to our colleagues and our fantastic working culture.
Inclusion & Accessibility
At ONS we are always looking to attract the very best people from the widest possible talent pool, and we are proud to be an inclusive, equal opportunities employer. As a member of the Business Disability Forum and a Disability Confident Leader we’re committed to ensuring that all candidates are treated fairly throughout the recruitment process.
As part of our application process, you will be prompted to provide details of any reasonable adjustments to our recruitment process that you need. If you would like to discuss any reasonable adjustments before applying, please contact the recruitment team in the first instance.
If you would like an accessible version of any of the attachments or recruitment documents below or linked to in this advert, please contact the recruitment team who will be happy to assist.
Things you need to know
Selection process details
Security Clearance
The requirement for SC clearance is to have been present in the UK for 3 consecutive years immediately prior to applying.
A lack of UK residency in itself is not always a bar to security clearance, but the Department will need to consider eligibility by exception on a case-by-case basis. You will be asked to provide information regarding your UK residency during your application, and failure to provide this will result in your application being rejected.
If you are unsure that you meet the eligibility above, please read the information available on Gov.uk on this link, or contact the recruitment email on the advert before applying to discuss, as failure to meet the residency requirements will result in your security clearance application being rejected and any offer of employment being withdrawn.
At the point of SC application, you will need to provide or give access to the following evidence:
- Departmental or company records (personnel files, staff reports, sick leave reports and security records)
- UK criminal records covering both spent and unspent criminal records
- Your credit and financial history with a credit reference agency
- Security Services records
Applicant support
Applicants may utilise AI resources in their submissions; ensuring all information provided is factually accurate, truthful, and original, avoiding any form of plagiarism to maintain the authenticity and credibility of the application process.
Stage 1 – Application & Sift
Assessment at application stage will be based on your CV, work history, skills, experience and the responses you provide to the two questions in the application form.
The two questions in the application form will have a limit of 500 word limit for each response.
Your CV should be tailored to demonstrate any skills, knowledge and experience that are relevant to the content of the role detailed in the essential skills. More details are found in the role profile which is attached.
In cases where there is a high number of applications the sift pass mark may be adjusted and candidates will be invited to interview in merit order, i.e. those scoring the highest.
Stage 2 – Interview
Should you be invited to interview, you will be assessed using various assessment techniques aligned to the Civil Service Success Profiles framework, where you'll be assessed against all the behaviours outlined in this advert.
Interviews will be conducted by Video Conference.
A reserve list may be held for a period up to 12 months from which further appointments may be made.
We often have similar roles available at different grades. If a candidate is suitable for a similar role or a lower grade than they have applied for, we may offer the candidate that role without the need to go through a further selection process providing the role has the same behaviours and essential skills.
Important Dates:
Sift date: From 16/10/2024
Interview dates:From 04/11/2024
For the full terms and conditions of the post, please see attachment below.
Feedback will only be provided if you attend an interview or assessment.
Security
Nationality requirements
Working for the Civil Service
We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
Diversity and Inclusion
Apply and further information
Contact point for applicants
Job contact :
- Name : Government Digital and Data Recruitment Team
- Email : Government-Digital-and-Data-Recruitment@ons.gov.uk
Recruitment team
- Email : Government-Digital-and-Data-Recruitment@ons.gov.uk
Further information
