U.S. flag An official website of the United States government
Dot gov

The .gov means it's official.
Federal government websites always use a .gov or .mil domain. Before sharing sensitive information online, make sure you're on a .gov or .mil site by inspecting your browser's address (or "location") bar.


This site is also protected by an SSL (Secure Sockets Layer) certificate that's been signed by the U.S. government. The https:// means all transmitted data is encrypted - in other words, any information or browsing history that you provide is transmitted securely.



software icon


Other Types:

data cloud icon


idea icon


design icon


scientific icon


technology icon


entrepreneurship icon


In a software and app development challenge, an organization asks solvers to create a software application to solve an existing problem or draw attention to potential uses of available datasets. A challenge may ask its solvers to use particular datasets or adhere to a set of outlined functionalities in their application. Such challenges provide open-ended paths toward a solution in order to encourage diverse and innovative participation. Desired solutions tend to be scalable and often require web-based assets. These challenges do not set out to solve extremely specific "big data," image processing or scientific problems, but could certainly use aspects of them all to develop a robust and scalable application. Challenges may also involve some algorithm development or data analysis.

A software and app development challenge often results in one of three distinct types of applications, depending on the needs of stakeholders.

  • End-user applications are tailored to the specific needs of a user, often the general public.. Also called front-office applications, they are the most common application referred to when the term "app" is used. Typically, "app" refers to a software application designed to operate on a mobile device such as a smartphone or tablet. App challenges are some of the most practiced and well-understood among the types described in this toolkit.
  • Back-office applications are not visible to the end user of an organization's product or service but will be accessed by those internally involved in a specific operation. A back-office application can run a company's inventory, financial records or manufacturing processes, and its interfaces are not generally designed for the general public.
  • Enterprise applications are complex, distributed and often highly scalable. They are used in large organizations and contain mechanisms that affect large groups of employees or the public. They interface with large, diverse user groups and must remain flexible to allow system administrators to effectively manage them based on their organizational qualities.

Key Takeaways

1. Clearly convey any solution and submission criteria.

The technical specifications for submissions likely will be different depending on the intent of your challenge. It could be designed to promote the use of government data for nongovernmental purposes, or you may want the government to use the final solution. In both cases, system security and access requirements will come into play.

Participants may desire detailed guidance in the following areas:

  • Operating systems and hardware: Generally speaking, the challenge rules should specify which operating systems (e.g., iOS, Windows, Android, web-browser based) and devices (e.g., iPhone, iPad, Android, Windows tablet or phone versions) you are able to support for testing.
  • Access to solutions: Requiring submitted apps to be available in an app marketplace such as Google Play or Apple App Store will facilitate easier access to the apps for testing and judging, but this is not a good solution if you wish to examine source code as part of the evaluation. If you do require submission in an app marketplace, request free access for those involved in reviewing and judging apps. The solver should provide a discount code or make the app available to everyone at no cost.
  • Functionality: Consider stating all requirements for functionality, including whether the submission must work while offline, online or when running on a large-bandwidth network.
  • Integration: If your software challenge is intended for use by your organization, it is essential to have active participation from any IT groups within the organization that will be responsible for integrating the software. This is especially true if it is an enterprise solution.
  • Specifications: Consider what other criteria you expect the submissions to address such as usability, privacy and security controls, and accuracy of content.

It's easier for a participant if you list all expectations, even if they seem trivial, as part of the submission requirements and judging criteria. Developers need to understand up front how their submissions will be evaluated. Some challenges target a near-perfect final solution, while others focus more on creating prototypes or software in an earlier stage of development. A successful challenge will balance the technical criteria of the challenge with an appropriately sized prize purse and development timeline.

2. Consider releasing solutions as open source.

It is often beneficial to require solvers and challenge platform vendors to deploy winning solutions to an open-source repository such as Github. This provides many benefits, including exposure, continuing development, accessibility and security. If you decide to follow such a strategy, you should be careful about the selection of the open-source license used. Some license types may pose an undue burden on developers or limit potential commercialization of derived products. To learn more about the relative merits of different open-source licenses, there are several sources. The Open Source Initiative provides general information about these license types . Your agency may have an attorney who can guide you in license selection. In addition, the White House has released guidance documents to support improved access to custom software code developed for the federal government.

3. Consider these best practices for getting the most out of an app challenge.

Mobile device applications are a rich area for challenges. Agencies often will turn to an app challenge for their first competition. Apps can help your agency call attention to its own data assets or focus attention on a problem. After several years of running app contests, the community has discovered several key learnings:

  • Engage the right communities and split the development process: The communities of solvers who do conceptual design (e.g., app task flow and graphical user interface (GUI) design) are usually different than the communities of solvers who code and know how to implement the app. If your app challenge includes the use of specified data as a requirement, keep in mind that neither of those communities may understand the data or the ensuing knowledge that is gained by using the data. To mitigate this issue, consider providing additional background information in plain language on the challenge website. Conducting one challenge where the same community does both tasks may not result in the most innovative and satisfactory solution. A better approach may be to split this into two challenges — one that results in a conceptual design implemented in a clickable prototype and another to realize the conceptual design in a delivered app. Depending on how you structure your challenge, you might be able to use winning ideas from the conceptual design phase to drive innovative functionality requirements in the build-out phase. Some crowdsourcing platforms have methods to break a challenge into both conceptualization and development contests. If it is not possible to separate these processes, you will need to work diligently in your communications and outreach to find those potential solvers who have all of the needed skills to participate.
  • Pick the right venue/platform: As mentioned above, some online crowdsourcing platforms are able to split your challenge into conception and development phases. Many platforms also have active communities of users who are looking for exciting opportunities to show off their app development skills and compete for prizes. A virtual or in-person hackathon can be a great way to motivate solvers working on your problem. The growing trend of hosting 24- or 48-hour in-person hackathons on college campuses or in local tech spaces can be a fun way to get people together in a room to build out potential app solutions in real time. If there is already a meeting scheduled where your targeted solver community will come together, consider planning an event around that meeting. An example of this is the Zapping Rachel challenge, which the Federal Trade Commission launched at DEF CON, one of the largest hacker gatherings.
  • Sustain and maintain solutions: Build elements into your app challenge that will help the apps thrive beyond the life of your challenge. If you are providing data for the apps, your agency must continue to maintain the data. Consider holding more app challenges to maintain the community of developers interested in your problem space. Some apps may take time to become fully developed and widely implemented. Solvers may benefit from additional assets from your agency including promotional support and technical assistance after your challenge ends.

4. Engage the business process stakeholders, especially for enterprise software challenges.

All challenges need close and ongoing participation from end users to prevent a mismatch of technical criteria and capabilities following a successful challenge. This is especially true for enterprise software solutions, which often target various parts of the software design lifecycle through a campaign of smaller contests. To be successful in integrating all of the parts, close participation is needed from all stakeholders who the software will support. This means business process owners as well as users. Basic architectural attributes such as rules engine decision drivers and decomposition of enterprise services are key factors to extensibility of the delivered software and need to be reviewed by all stakeholders. Upon completion, business process owners should be able to quickly exercise all of their use cases against the software to assure utility.

The challenge should identify stakeholders from every aspect of your line of business who have the knowledge to evaluate the delivered solution's utility against their specific enterprise use case to realize useable results. Stakeholders should be involved early and often, even before the launch of a challenge, to ensure requirements are clear and appropriate for the enterprise. Your agency should plan on significant and ongoing organizational overhead to manage the development and integration of enterprise software solutions, more so than for end-user or back-office apps.

5. Ensure you can evaluate software submissions in a safe and secure manner.

Consider how you will test and evaluate the submitted software. Working with relevant experts at your agency, you should determine how submissions will be tested and what access they will have to your network. Work with your chief information officer or other appropriate staff to ensure there is a safe location that provides necessary security for you to test the submissions without endangering your system network. You also should consider necessary precautions for your judges if they will be using devices to test the submissions. Typical solutions include using devices specifically set aside for testing, creating a testing network isolated from your agency's online assets and using anti-virus software to help protect your systems from being exposed to malware and viruses. Consider these best practices:

  • Only receive source code as a deliverable. Most malicious code is inserted in compiled code.
  • If possible, perform static code analysis on the software and ensure any significant threats are addressed. Some vendors provide this as a service prior to code delivery.
  • If possible, perform quality reviews of code submissions. Some vendors provide this as part of their service.
  • In cases where security is crucial, it may be necessary to have the deployed system analyzed for threats by a security service. There are crowd-based communities of white-hat hackers that can be tapped to find security threats and recommend improvements.
  • Provide environment-neutral resources on which teams can demonstrate their solutions to an audience without interfering with local configurations or compromising venue IT resources. An example would be to use a multitude of video inputs to a projector rather than a single PC and share drive or USB stick.

Still have questions?

Contact the Team

Case Studies