Senior Data Operations Engineer
OFGEM
Apply before 11:55 pm on Monday 18th November 2024
Details
Reference number
Salary
Job grade
Contract type
Business area
Type of role
Digital
Working pattern
Number of jobs available
Contents
Location
About the job
Job summary
It is important that we become a fully digital regulator so that we can transform the energy sector. We are therefore embarking on a high-profile data and digitalisation programme, and it is within this scope of work that you could find a career-defining challenge.
Our team is working with the most cutting-edge cloud and data technologies, so this is a great opportunity to further your knowledge with the support of experienced colleagues. As such, you will help deliver our new Strategic Common Data Platform, which will connect internal and external data sources. Taking responsibility for designing and developing Azure pipelines (which will include data transformation and cleansing), you will create accessible data sets that can be easily distributed and analysed. You will also engineer manual data flows that are both scalable and repeatable, and you will document source-to-target mappings. Across all that you do, you will align your projects and datasets with Purview.
This is an exciting opportunity to network extensively with both internal colleagues and external partners. You will therefore be a natural fit in our team if you have good communication skills and you can bring a proactive mindset to identifying and resolving issues in databases, and data processes, products and services. With a background of working with Azure Data Lakes, data warehousing, and storage accounts, you will be capable of building all pipeline facets including data interfaces. Proficient in in T-SQL and Python, you will be able to transform data into appropriate storage formats, engineer data for analytics and devise error handling.
Ofgem relies on having a workforce that reflects the society we serve, so we welcome candidates from all backgrounds, and especially those from underrepresented groups. While this is a full-time role, flexible working patterns and job shares are welcome, and we offer a fully hybrid environment. We’ll also support you with excellent training and development opportunities, plus a competitive pension and generous annual leave.
Job description
We play a key role in supporting Ofgem to be a dynamic, inclusive and high performing organisation, both striving to continuously improve our efficiency and effectiveness in meeting the needs of the organisation, and also setting clear corporate standards and controls in order to ensure the organisation operates within our statutory and legal requirements, and good practice standards.
The Data Operations Engineer is an expert role to help accelerate data driven decision making, digital transformation and innovation across Ofgem through the provision of high quality, timely and valuable data and analytics in a resilient and consistent manner.
The role will be key to achieving Ofgem’s vision of Digitisation and supporting the innovation and growth agenda of:
- Making OFGEM a recognised digitalisation leader with innovative approaches in the exploitation of data using emerging technologies
- Establishing the capabilities to focus on specialist areas including data analytics/insights, data innovation, artificial intelligence, data sourcing and automated data pipelines
- Training our workforce to be comfortable and skilled in understanding, using and asking questions of data, visualisations and analytics output
- Achieving cost efficiencies in targeted areas e.g. automate data preparation and processing, automate production of commonly used reporting, share data live online
This role oversees the service provided by our modern cloud data platform. This platform includes; data ingestion, data processing, data quality improvement*, master data management*, data modelling*, data science* and data visualisation*.
Items with a * have peer team leads responsible for specifying and leading development of new content but this role could help deliver and will be responsible for ongoing running of our end to end data analytics pipelines.
This role will work with the Head of Data and AI Enablement to monitor our end to end service, establishing trust in the data and analytics provided and will aim to optimise the usefulness and value it provides.
Your initiative and ideas on how to measure, grow and improve the following will be required:
- Levels of usage of data/data products e.g. user session count, user session duration
- Levels of value provided by each data product e.g. dashboard
- Levels of value provided by each underlying data set e.g. source file
- User trust and satisfaction e.g. net promoter scores or review feedback
This role works closely with the Head of Data Management and Operations to deliver priority improvements, oversee current work and improve data platform service levels. This along with the proposed benefit of each proposed action will help this role holder demonstrate service value ongoing as well as the planned functional improvements each work package will deliver.
Person specification
Key Responsibilities
Delivery of set of team deliverables, and managing and co-ordinating successful outcomes across team members and collaborating teams
- Sets clear and transparent work objectives, milestones and success metrics at team and individual level to oversee and co-ordinate successful team outcomes.
- Collaborates closely with other teams to manage interdependencies, risks and resourcing to support portfolio delivery.
- Jointly responsible with the PDL on welfare and pastoral care of all colleagues.
- Demonstrates effective diversity and inclusive team management within their team and the wider organisation.
Essential Criteria
- Experience monitoring impact, service quality or value delivered from work, platform and service (LEAD)
- Experience using and building modern cloud data technologies (e.g. Microsoft Azure Data Factory, Microsoft Azure Synapse, Microsoft Azure Machine Learning, Microsoft Power BI service) to deliver resilient and scalable data pipelines (LEAD)
- Demonstrable experience in data analytics, data technologies or development processes
- Experience prioritising development time, effort or financial resource across a portfolio of work
- Clear written and verbal communication at a senior management level
- Experience of using master data management techniques to improve data quality and integrity
Desirable Criteria
Experience in:
- Service design or implementation
- DevOps processes and methodologies
- Agile/scrum process and methodologies
Experience in:
- ETL development programming in Python and Pyspark
Behaviours
We'll assess you against these behaviours during the selection process:
- Changing and Improving
- Working Together
- Delivering at Pace
Technical skills
We'll assess you against these technical skills during the selection process:
- Please refer to the Candidate Pack and Role Profile attached for full details.
Benefits
Things you need to know
Selection process details
When you press the ‘Apply now’ button, you will be asked to complete personal details (not seen by the sift panel), and upload a copy of your CV anonymising all details where necessary.
You will then be asked to answer 3 Technical Questions evidencing how you meet the essential and desirable skills and capabilities listed in the role profile. Please ensure you demonstrate clearly, within these answers how you meet each of the essential and desirable skills and capabilities.
The Civil Service values honesty and integrity and expect all candidates to abide by these principles. Ofgem take any incidences of cheating very seriously. Please ensure all examples provided are of your own experience. Any instances of plagiarism or other forms of cheating will be investigated and, if proven, the relevant applications will be withdrawn from the process.
Feedback will only be provided if you attend an interview or assessment.
Security
Nationality requirements
Working for the Civil Service
We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
Diversity and Inclusion
Apply and further information
Contact point for applicants
Job contact :
- Name : Jennifer Hughes
- Email : Recruitment@ofgem.gov.uk
Recruitment team
- Email : Recruitment@ofgem.gov.uk
