Job Specification

Purpose of Position :

To manage and co-ordinate the full range of l&IT system testing services of the ministry's very large and complex Road User Safety Modernization Project (RUSMP) that will deliver multi-platform environments to support critical program business needs for the Road User Safety Division in both legacy and web-based technology. To be responsible for systems testing, coordination and support for User Acceptance Testing (UAT) and to direct/oversee the provision of testing and related services by partner branches (Road User Safety Solutions Branch) and third party providers. To establish and maintain a testing framework that supports (automated) software testing and ensures testing integrity and best practices. To coordinate, participate and shape the testing strategy and the development of testing methodologies, standards and tools. To provide I&IT senior management and specialists with testing expertise, strategic advice, recommendations and collaborative I&IT solutions for application development and maintenance issues discovered through testing. To provide technical direction to project staff and consultants, and ensure cost effective and efficient testing services.

Duties / Responsibilities :

l&IT testing includes a wide variety of testing types, such as: sanity testing, configuration testing, security and access control testing, user interface testing, functional testing, data and database integrity testing, business cycle testing, Accessibility for Ontarians with Disabilities Act (AODA) testing, BI (Business Intelligence) testing, web service testing, performance profiling, load testing, volume testing, failover and recovery testing as well as user experience testing. The accountability of these testing types will rest in this position.

1. I&IT Applications Development and Maintenance Testing

• Provides leadership and management for the cost-effective testing of all applications in scope of the RUS Modernization project at all stages of the Systems Development Life Cycle (SDLC), including testing requirements and design, enhancements and modifications, training and support, and the provision of authoritative expert advice and recommendations on remedies for risks and problems.
• Oversees and ensures the integrity and quality of testing processes, techniques and project plans, (e.g. for developing test cases; defining and developing testing automation; testing applications and documenting results; anticipating and resolving (the potential for) incidents and/or problems; costing and evaluating options).
• Leads and supports testing-related research and development, including tools and methodologies, best practices and industry innovation and uses, in order to provide technical leadership in the development of cost-effective, efficient and appropriate testing services and expert recommendations.
• Establishes and maintains a testing framework that encompasses a set of processes, guidelines, concepts, components and tools that support (automated) software testing and reflect best practices. Ensures consistency, reusability and integration of testing tools, techniques and outputs with related Project/Cluster/enterprise l&IT standards and requirements.
• Manages business risks related to technology services and access by proactively identifying issues and threats presented by testing (e.g. loss of application integrity, functionality and/or user access), and ensuring testing plans and processes are implemented to manage risks.

2. Project Management

• Directs project planning for all testing phases of the RUS Modernization project; establishes priorities; schedules and coordinates work of staff and/or consultants; ensures compliance with enterprise and corporate l&IT strategy.
• Develops and identifies project resource requirements and negotiates to secure them.
• Supervises multidisciplinary teams of project staff including I&lT staff, client staff and consultants.
• Directs the full deployment of l&IT testing processes and systems in partnership with solution development teams within the RUS Modernization project, Road User Safety Solutions Branch (RUSSB), Information Technology Services (ITS) and/or external service providers, ensuring all considerations are accommodated (e.g. clients'business requirements, operational continuity requirements, risk containment, user access, etc.).
• Develops performance measures and reviews to evaluate testing results for each application, and ensures the conceptual integrity, reliability and validity of testing tools and processes (e.g. from initial identification of goals and objectives though to selection and use of specific testing types, tools, performance benchmarks, sequencing/phases, processes, remedial work, final approval, and documentation).
• Ensures testing is conducted according to established budgets, schedules and standards.
• Conducts follow-up with l&IT applications Development Manager and/or System Solutions Manager, l&IT specialists and clients to ensure testing results and recommendations are understood and adopted, and provides expertise to support related change and implementation.

3. Expertise/Relationship Management

• Provides testing process and content consulting expertise for the RUS Modernization project and works collaboratively with other Cluster managers to support mutual alignment and integration of technical processes and content.
• Develops and manages effective relationships with business clients, other clusters/ministries and enterprise level specialists, suppliers, OPS team members and external clients, stakeholders and the industry; understands their business needs, and relates to their business solutions development and maintenance needs (e.g. in the areas of architecture, data, information, systems technology, services and associated supports and tools), and ensures consistency in the delivery of testing services to meet those needs; develops/recommends co-operative approaches to resolving testing-related issues (e.g. user access during testing); and discusses/evaluates technologydevelopments and trends that may be applicable.
• Participates on various project and OPS committees to develop/revise testing-related I&IT policies, standards and guidelines across the client ministries, Cluster and the broader OPS. Leads and manages the development and implementation of testing standards, policies, guidelines, procedures and methodologies to ensure Project/Clusterwide consistency, integration and re-use.
• Participates as an expert testing consultant on project and Cluster committees and task forces related to business solutions information, systems, technology and services to improve project effectiveness in program delivery.
• Provides strategic input to l&IT strategic and associated operational plans by recommending and influencing new l&IT testing strategies and initiatives to align with project and Cluster strategy, satisfy corporate goals and meet client business needs.
• Provides expertise and specialized input into the development and/or acquisition of testing tools, training, etc. for l&IT professionals, project and Cluster staff and internal/external users of business solutions applications. This includes the use of hardware and software acquired, developed and/or implemented, adherence to standards, etc.
• Develops and provides updates to testing content for the project l&IT Service Catalogue (e.g. testing service types)

Knowledge :

Comprehensive expertise in I&IT applications testing principles, standards, methodologies, tools, techniques and trends, to provide direction, leadership and authoritative testing expertise concerning the project's diverse core l&IT applications and to oversee and ensure the quality and integrity of testing by contracted third parties.

• Comprehensive knowledge of applications-related hardware and software, networks, systems analysis, design, development, implementation, installation, operational techniques and methods for technology, data, applications, networking, and/or security, to research, plan, lead and evaluate both interdependent and independent, complex testing activities; make authoritative recommendations and contribute expertise to project management and general management decision-making; anticipate/diagnose serious problems and provide viable options and solutions; prepare project testing policies, standards and procedures; and manage vendors' performance.
• Comprehensive knowledge of the application environment; test tools, processes and standards for integration, functionality and usability testing; user acceptance and release test procedures; scientific test analysis methodology, etc., to ensure the integrity and completeness of testing for all applications, ensure consistency and alignment of apptesting solutions.
• Information systems development and maintenance expertise, and related quality assurance and change management processes, methodology and tools, to plan, lead and evaluate testing needs assessments; ensure testing does not introduce new risks or fail to identify risks in diverse, multi-site applications that are either interdependent or independent; support change management; and provide authoritative consulting expertise and support.
• Knowledge of application architecture to lead testing to ensure consistent, integrated applications and associated standards and tools.
• Knowledge of systems methodology and problem-solving approaches to provide cost-effective testing expertise and advice to senior management and users, and technical direction to I&IT specialists.
• Knowledge of l&lT technology, hardware and software, particularly those with specialized solutions, to understand and capitalize on the advances in l&IT and their relevance and potential for cost-effective and efficient applications testing and projects.
• Knowledge of the systems development life cycle (SDLC), business and information systems and analysis, systems development/maintenance, systems management, data management, project management methodologies, techniques and best practices to ensure effective delivery of testing services that support the optimization of the project's I&IT investments.
• Experience in large-scale portfolio and project management, encompassing widely accepted project management practices, and innovative and advanced practices, including those with more limited, specialized applications; project integration management; scope management; time management; cost management; quality management;; risk, budget and procurement management; and related tools, techniques and methodologies in order to ensure effective alignment of the testing function; provide leadership for testing project managers regarding current project management trends, standards and processes; and participate in managerial planning, reviews, problem-solving and decision-making.
• Knowledge of relevant legislation; l&IT standards, processes and conventions used in the OPS/Cluster (e.g. ITIL, Remedy GO-ITS), integrated planning cycles, guidelines and procedures; and strategic and operational planning methodologies, to ensure alignment and compliance in testing plans, policies, projects and recommendations.
• Knowledge of architecture standards and methodologies; systems analysis, design, development, testing and implementation methods; and broad knowledge of technology, data, applications, networking, and security, to provide authoritative testing advice to technical project managers regarding multiple, complex, large-scale testing projects.
• Project management skills to plan, lead, co-ordinate and oversee concurrent specialized testing projects, services and activities.
• Leadership skills to supervise project leads, systems specialists and support staff, provide motivation and foster teamwork.
• Program and portfolio management skills to plan, integrate, deliver and support testing work.
• Risk identification skills, to anticipate and neutralize potential risks resulting from testing and related changes; identify, assess and make authoritative recommendations on risks identified through testing.
• Consultation, negotiation, communication and client service skills to act as liaison and expert advisor with all levels of senior management and clients.
• Ability to manage contractor/vendor relationships.
• Proficiency with PC-based common office software to schedule, track and report on projects; create analyses, reports and recommendations; make presentations; estimate costs; produce correspondence and co-ordinate activities.roaches, discuss and evaluate testing tools and trends, and share best practices; and provide expert technical

Judgement :

Work is performed under the general direction of the Senior Manager PMO, Quality & Support, and within the framework of the mandate of the OPS enterprise; the Cluster and its client ministries and their business needs, objectives and policies; OPS I&IT strategies, plans, standards and policy; and related industry standards and best practices.

The position exercises independent judgment in:

• Leading the formulation, cost-effective and efficient delivery of l&IT applications testing and associated services to meet project/Cluster/clients' very diverse requirements, in accordance with varying provisions in Service Level Agreements, OPS/Cluster/project priorities, policies, standards and available resources.
• Providing functional expertise as input to strategic plans and policies; integrating testing planning with project and Cluster strategic planning; ensuring testing is consistent with OPS policies and standards and overall approaches, needs and systems; and providing related consulting expertise and support for very diverse applications/client needs across the province.
• Balancing resources to meet conflicting demands cost-efficiently and effectively without sacrificing quality.
• Managing risk in the identification and resolution of testing problems; and supporting the development, co-ordination
and implementation of solutions to relieve existing/potential problems and avoid the potential creation of further
problems.
• Recommending additions, enhancements and changes to architecture, systems, hardware, software, data
management, tools, methods, etc. that improve the cost-efficiency of project testing without negative impacts on
clients' businesses; service future business needs; and also ensure integration and alignment with related l&IT.
• Leading the development and implementation of a comprehensive framework of testing standards, policies and
procedures which support diverse, complex and critical testing requirements, comply with legislation, and are aligned
with related project/Cluster/enterprise frameworks.
• Providing input to strategic and operational l&IT plans and policies; recommending strategic and technological
testing and related directions that are aligned with enterprise directions.
• Translating conceptual and logical architectural models to physical implementation for testing purposes, and in
advising on solutions for problems discovered through testing.
• Resolving testing-related client service/service delivery issues in complex, multi-project environments across many
sites.
• Leading, co-ordinating and developing highly specialized staff and managing the professional interface with related
functions across the project.and Cluster.
• Setting challenging but consistently achievable standards for testing services.
• Providing contingency and emergency plans, and ensuring provisions are ready to be activated

Accountability - Programs :

Managing all l&IT applications testing for the project and the clients (ministries, enterprise-driven solutions), to ensure cost-efficient and effective testing services, expert advice and support that ensures risk-managed applications that can meet very diverse client needs cost-effectively and efficiently across the division.

• Supporting the Cluster/Branch with technical content and process expertise.
• Establishing/maintaining technical l&IT testing performance standards and measuring performance against those
standards.
• Providing strategic recommendations, including risk, opportunity and priority assessments to support high priority
testing projects and other key initiatives.
• Ensuring all testing plans, projects and initiatives are integrated and consistent with corporate l&IT policies,
procedures and strategies.
• Monitoring, evaluating and reporting on progress, and taking pre-emptive steps to ensure barriers are removed and
progress/performance reflects plans.
• Providing testing-related policies, standards and tools.
• Providing/activating contingency and emergency plans, and ensuring provisions are ready to be activated at any
time.

Accountability – Personnel :

Responsible for providing team guidance, leadership and direction. Coordinates the work of, and provides technical expertise to staff, consultants and other resources assigned on a project basis. Also responsible for the selection, provision of technical advice and adherence to quality standards of consulting staff retained for the RUS Modernization project

Accountability - Finance and Materials :

This position is not directly accountable for budget/financial management but it is accountable to ensure that all project resources and assets assigned are used within allowable government policies and directives and assessing cost/benefits of recommended technology/service changes.

Accountability - Impact of Errors :

Poor judgment in implementing l&IT testing services for the RUS Modernization Project will result in the implementation of ineffective and costly solutions that will not meet client business needs, disruption to service delivery of critical program and business operations that could cause costly long-term implications internally and/or across other
ministries and external partners.

Contact - Internal :

• Frequently with the RUS Modernization project management team to provide leadership and authoritative subject
matter expertise; plan and manage application testing functions; and to participate in strategic/operational/project
planning, reviews and problem solving.
• Frequently with Cluster/client ministries' senior management to identify testing and related needs; provide expertise
and influential recommendations; participate in negotiating service level agreements; obtain service feedback and
report project progress performance; anticipate and provide authoritative expertise and leadership to resolve major
testing risks and problems.
• Frequently with project and Cluster internal stakeholders, partners and users to consult and identify needs, obtain
feedback, and ensure the provision of appropriate testing services and expertise.
• Regularly serves as member of Cluster/enterprise committees to develop testing and related standards and tools,
and promote integration and co-ordination.
• Regular contact with counterparts throughout the Cluster and OPS and at the enterprise level to discuss and share
plans/concerns, develop common testing solutions/resolve shared problems, and ensure alignment.

Contact - External :

• Frequently with consultants, service providers and vendors to plan and evaluate acquisitions and services to support
the testing function and monitor contractors and services.
• Regularly with Project/Cluster and clients' external stakeholders and partners to consult, identify needs and issues,
plan, lead testing and evaluate results to ensure functionality, cost-effectiveness, efficiency and risk-management of
business solutions that comply with policies and standards.
• Frequently with l&IT industry professionals and professionals/peers in other jurisdictions and the private sector to
remain current on pertinent trends, developments, best practices, tools and methodologies.
• Frequently with other governments to share experience with testing tools and techniques, and develop partnerships/co-ordinate joint testing where appropriate.