In this phase you'll determine the challenge structure and implementation timeline. During this critical step, you'll work with internal groups to establish eligibility and submission requirements, terms and conditions, and judging criteria. You'll connect with your communications team to outline your announcement and ongoing outreach strategy to engage potential solvers.
Create an Implementation Plan
You've worked through your challenge structure, judging process and communications plan. Now you have to figure out how to get it all done.
Create an implementation plan that consolidates the activities and milestones identified through these previous steps to make sure you have enough personnel assigned to carry out the challenge effectively.
Think of it as your project management plan.
During this step, you'll also decide whether implementation will be carried out completely through government resources or if you need to establish contracts or partnerships for parts of your challenge. When challenge management is outsourced, consider making metrics data collection and results analysis a requirement of the management entity. Also, be sure to clearly outline roles and responsibilities between organizations.
Before publicly announcing your challenge, you'll need to finalize any contracts, partnerships and agreements you have created to implement your challenge. Never underestimate the time it will take to put into place interagency agreements or other contracts.
To create an implementation plan, you must take a closer look at how the structure, judging process, communications plan and other details come together to facilitate a successful challenge.
Ask the following questions to begin fleshing out your implementation plan:
- What resources will you need to actually launch the challenge?
- What, if any, requirements are there on data quality, quantity and security for various phases of the challenge?
- In many cases a challenge website serves as the interface between organizers and the solver community and requires careful design considerations. What features does the site need to accommodate registration, data access, submission of solutions for evaluation and other aspects of participation?
- Should you perform a test run of the challenge website or the competition itself before public launch? What resources do you need to make this happen?
- How many support personnel (FTEs) will you need to carry out the challenge?
- What is the process for collecting data or performance results during the course of the challenge? Will you need to draft an operating procedure for data or sample collection?
- What training aides, tutorials, frequently asked questions or other learning materials should you provide to participants?
- How will you present feedback to participants? Will you have a leaderboard, and what resources will you need to support it?
- Will you have a discussion forum or blog available throughout the challenge for contestants to ask questions of the organizers?
- Will you allow participants to team up during the challenge and provide mechanisms that make it easier for them to do so?
- How will you address any disputes about judging?
- How will you announce winners and distribute prizes?
- How will you disseminate results of the challenge?
- How will you store any reports and scorecards?
This toolkit contains a lot of information on identifying a legal authority and the requirements that come with it. Make sure you understand the appropriate legal authority and how it affects the implementation of your challenge.
You may be able to carry out the entire challenge using government resources, or you may want to hire a vendor to handle all or part of it. It could take six months or longer to issue a contract, so factor this into the overall project timeline. The decision to use a contractor can also be affected by the terms and conditions of whoever puts up the money for the prize purse.
When using contractors, set objective performance milestones to enforce the implementation plan and create specific metrics you can use to measure success.
The challenge team is responsible for implementing, operating and monitoring the challenge throughout the process. Designate a project leader and make sure all other team members know and understand their roles and assignments and how it all fits into the challenge timeline.
The project lead should be familiar with all aspects of the challenge and ensure that the skillsets of team members match their assignments. It's critical that you choose people with appropriate expertise for key aspects of the challenge.
Think about it. Do you really want to rely on legal advice from someone who isn't an attorney? Of course not. So it stands to reason that you'll want an actual IT specialist to provide any technical assistance and a communications expert to handle promotion and outreach. Also, make sure your challenge lead can carry out your entire challenge plan and not just pieces of it.
If it makes sense, consider a platform manager who can support challenge infrastructure and data scientists to help with metric preparation and data analysis.
Use this checklist to make sure you're on solid technical footing:
- Make sure the challenge website provides all the information participants need. This may include background information about the challenge, the reason you're running the challenge, rules/eligibility for participating and winning, evaluation criteria and any other pertinent details. The site also should provide access to tutorials, data and other resources that create a level playing field for all participants across all phases of the challenge. This includes any access to training, a leaderboard and tests.
- Have a plan for helping participants who experience difficulty accessing the website, required data, evaluation platforms or anything else needed for the challenge. You may want to establish an email address or phone number participants can use for such support.
- Have a back-up plan in case your system goes down. Also, have a plan to recover from a system failure.
- Be ready to scale resources up or down in case of high or low participation to make the most efficient use of resources during the challenge.
- Obtain a realistic cost estimate for the IT platform or cloud resources before the project begins.
- Establish a process to pay winners. Will payment come from the agency, or can a partner organization or contractor pay the winner?
Reporting is a necessary part of any government project, and a challenge is no different. You'll at least need to provide summaries to your agency about a challenge at various stages along the way and afterwards.
Create interim and final reports to capture the totality of your challenge experience. These reports give you an opportunity to review progress and examine milestones, particularly if you're using contractors. Decide what information to collect and include in these reports, who will put them together and for what audience.
These reports also will come in handy when the White House Office of Science and Technology Policy and General Services Administration comes seeking information for the next biennial progress report on challenges and prizes to Congress.
2.6 Create an Implementation Plan (current page)