In this phase you'll put your plans into action. You'll roll out your communications plan, accept submissions and interact with solvers to generate interest and enthusiasm throughout. After your submission deadline, you'll begin to evaluate entries, select winners and verify their eligibility. Successful execution of this phase is critical to maintain credibility of the challenge and your agency.
Your approach to accepting solutions from solvers will vary based on challenge type, legal authority and what's being submitted.
A submission could be a paper proposal, video, poster or code provided as a link or attachment. It also could be physical hardware brought to a demonstration event.
Provide specific guidance to solvers in the submission requirements. They need to understand from the get-go what they'll need to do to submit. Consider proper delivery, storage and handling when actual hardware is involved.
During this step, you'll also begin to carry out the intellectual property decisions you made in your terms and conditions, which will govern how you can use the solutions. The judging process also can affect how you organize and store submissions.
And don't forget federal records requirements. Understand your agency's process for managing records and know how long you must maintain submissions as federal records.
When you accept solutions, you trigger the start of the judging process.
You may have a third-party contractor acting as your challenge administrator. If so, that contractor may be responsible for making sure all the submissions come in on time and meet requirements.
If you haven't hired an outside challenge administrator, then your internal team will have to manage the submission process.
It's best to plan for a large number of submissions and be prepared to tap additional judges if necessary. You may even get more than you can manage so have a risk mitigation plan ready in case the response to your challenge is overwhelming. It's unlikely, but it has happened.
Some challenges call for paper-based or online submissions. Others require participants to send in prototypes of devices. Set up a clear chain for handling submissions and make sure they can be accessed by challenge administrators, judges and others who need them.
Place appropriate restrictions on submissions to avoid cheating and unsportsmanlike conduct. An example: In managing the acceptance process for its International Space Apps Challenge, NASA requires all projects to have a profile added to the Space Apps website before they can be considered by judges. This process works for solutions developed at weekend hackathons around the world. All work is submitted via GitHub, where the solutions can be accessed for global judging.
You've given participants three months to submit solutions to your challenge. For two months and 28 days, you've seen only a trickle of activity from solvers.
If you're running your challenge as a completely virtual experience, you may even wonder if there is anyone out there working on a solution to your problem.
Don't stress. But get ready.
You're likely to see a deluge of submissions in the hours leading up to the deadline. Even if your submission period is open for several months, most solvers will submit in the final two days. That's just how it is.
Because of this, it's a good idea to set a submission deadline during normal business hours. You want to make sure your team members are available to assist solvers with any technical difficulties, questions and concerns right up to the final minute. This is easier to do at 3 in afternoon than at midnight.
If you've done a good job of implementing your communications plan, you should expect a large number of submissions. However, if you don't receive enough submissions for a robust competition, consider extending the deadline if your original rules allow for it. For this reason, it's good to build some flexibility into those rules and timeline.
Technical difficulties—the bane of modern existence.
Generally, accepting submissions through a web portal is consistent and efficient. But technical issues are inevitable.
Since most submissions arrive in the final hours, you need to have someone ready to handle questions, concerns and technical issues.
We'll say it again: Set your submission deadline for a time when the appropriate members of your team are available. And 11:59 p.m. probably won't work.
If you're asking for hardware or prototypes, be prepared for issues with shipping and receiving.
Your terms and conditions should specify eligibility requirements in accordance with the legal authority you're using. These requirements may cover a variety of concerns including indemnification and liability, insurance, and citizenship.
The broadest legal authority—first provided by the America COMPETES Act and updated by the American Innovation and Competitiveness Act—requires that, "in the case of a private entity, [it] shall be incorporated in and maintain a primary place of business in the United States, and in the case of an individual, whether participating singly or in a group, [he or she] shall be a citizen or permanent resident of the United States."
Have a clear process in place for participants to prove they meet eligibility requirements before they submit. This way participants won't spend countless hours on a submission that can't legally win, and your judges only have to evaluate submissions that can.
You can do this by including a self-certification in the submission requirements: "By submitting a solution, participants are self-certifying that they are eligible."
But there's no way around it. You'll still need to verify eligibility before you make the final awards.
The terms and conditions of your challenge set the eligibility and participation requirements for solvers. They also dictate your agency's relationship with the solutions, especially as it relates to intellectual property.
Your terms and conditions should specify how submissions will be handled, if and when they'll be made public and what will be done with them.
You may want to publish abstracts of all submissions for public voting. Maybe you consider all submissions open source and so release the entirety of them. You may even decide to publicize nothing about the submissions other than descriptions of those that win.
Each challenge is unique, so it can be difficult to predict which parts of solutions you want to make public and which should remain the intellectual property of the teams. You may even have to change your approach in midstream. Maybe you realize halfway through that certain information would be more helpful if it were made public but you didn't cover that in your terms and conditions. That's OK. You can still decide if you want to formally ask solvers for permission to use their intellectual property.
As a general rule, though, it's safest to stay true to your original terms and conditions. This ensures participants remain confident and comfortable with your challenge process.
It goes without saying: You should work closely with your agency's intellectual property counsel as you write your terms and conditions.
You may think you've done a first-rate job explaining technical details, but issues can arise after your challenge rules go public. And there's always room for interpretation.
Some challenges have well-defined requirements and ask for simple submissions. Others require teams to physically deliver prototypes of complex devices. For the latter, make sure those devices meet testing specifications so data can be fairly and accurately assessed.
Consider letting participants ask technical questions before they submit their devices to make sure they can be properly tested. You can even include this step in your communications plan.
You may even want to convene participant forums well before the submission deadline to allow unanticipated questions to surface in a fair and transparent way. You don't want miscommunication to be the cause of any disqualifications.
Record all questions you get from participants and include them in a "frequently asked questions" document. Sharing your answers with everyone maintains a level playing field.
Don't wait until the day of an in-person competition to determine how many competitors will be there. Set incremental checkpoints to gauge response to your challenge.
Require participants to register well before the competition. Ask for number of team members, proof of eligibility, etc. You also can set a deadline for work-in-progress submittals. Ask participants to provide the current status of their solutions with a photo, video or short written summary.
These checkpoints will help you plan for those who actually show up on competition day. Competitors who have made little to no progress are likely to drop out or not show up. And if they didn't register then they can't compete.