Certain generic questions emerge in the evaluation of secure products.  These are not particular to the voting setting, but can be used as the basis of an assessment methodology for electronic vote tabulation systems.  The list presented here can be augmented with additional items or finer detail, as necessary.  Purchasers of secure systems should work with independent testing agencies that have demonstrated the high level of expertise necessary in order to evaluate vendor responses to these questions.

1. What are the assets that require security protection?
2. What security risks have been identified, and what is the likelihood of each?
3. What countermeasures have been specified to deal with the identified risks?
4. What security assurance level has been selected for the system?  Justify the appropriateness of this rating.  How has conformance been established?
5. What assumptions are made about the operating environment in order for it to be deemed secure?
6. What are the policies and rules required to enforce security?
7. What are the specified security functions and assurance measures?  Have these been traced back to the functional requirements to insure that coverage is comprehensive?
8. Has a security requirements rationale document been presented?  Does it demonstrate consistency with the security objectives for the system?  Is the rationale comprehensive and consistent?  Are any objectives unsatisfied, and if so, why?
9. What are the integrity concerns, and how have these been addressed?
10. What procedures are in place for secure system development? How have these been enforced and documented?
11. What are the resource allocation, priority of service, and fault tolerance policies and procedures?
12. What are the data requirements, and how are these implemented and enforced?
13. What are the data retention policies and procedures?
14. Have all communication paths been identified and secured as appropriate?
15. What are the confidentiality requirements, and how are these implemented and enforced?
16. What are the user roles?  How are rules applied and enforced with the roles?
17. What are the authentication, authorization, and access control policies?  How are these applied and enforced?
18. What are the administrative tasks and responsibilities?
19. Have the interfaces been assessed as to their appropriateness and correctness?
20. Are all administrator and user guidance documents complete and useable?
21. What are the startup, shutdown, recovery, and rollback policies?  Which roles are responsible for these tasks?
22. How is the system delivered, installed, and generated?  Which roles do this?
23. What tests are performed in order to insure correctness?  When are these tests done?  Who is responsible for conducting these tests?
24. How is the system validated for acceptance and compliance?  Who does this?
25. What are the facility requirements, including physical protection of the system? What roles have been assigned responsibility for facility aspects?

The following questions can be used in conjunction with the generic security questions in order to elicit information regarding any electronic balloting and/or tabulation system under assessment.  Answers should include thorough documentation and independent evaluation and testing to support vendor claims.  Additional questions pertinent to the particular system being investigated should be added as necessary.
 

1. What means is used to separate voter identity from voted ballot?
2. How is the balloting process secured such that voter submissions can not be observed, or recorded in any way that is traceable to the individual voter?
3. What actions on the system are audited?
4. How is the auditing process precluded from associating voters with cast ballots?
5. How is the audit trail accessed and used?
6. Who is permitted to access the system (through all aspects of handling)?
7. What facilities are provided for recount purposes?
8. How are voters authenticated and authorized to cast ballots?
9. What access controls are in place to ensure single ballot per voter per election?
10. If multiple systems are deployed, how are voters tracked so the same person does not vote in different formats?
11. What controls are used to ensure that the correct ballot is provided to the voter?
12. What controls are provided to ensure that each ballot item is voted properly?
13. How are all forms of tampering detected and prevented?
14. How is vote confirmation provided without ballot-face receipt?
15. How is the voter prevented from retaining a copy of the cast ballot?
16. How does the system assure that each ballot has been correctly recorded?
17. How does the voter know that a cast ballot has been accepted?
18. How is vote tabulation correctness assured?
19. What features are employed to ensure operability of the voting system throughout the election?
20. How are downtimes handled in the event that they do occur?
21. What alternative balloting system is available for voters when the system is down?
22. How do the poll workers and system administrators know that the system is operating correctly?
23. How is the voting system precluded from use when deemed inoperable?