Software Evaluation Checklist: What are the Requirements for Evaluating IT Project

In the production of custom IT solutions, we appoint a manager who monitors the work and interacts with the customer, we form a team and we deploy the project environment.

The software product development process involves several phases. Each phase consists of three blocks: requirements specification, implementation of the solution, transfer of the project to the customer. Let's talk more about the stage of requirements specification.

Clarification of the requirements will help to clearly identify the needs of the customer, determine the resources and team, assess the value of the IT project. Based on the requirements, the development team agrees with the customer on the phases and timing of the solution and determines the methodology for acceptance testing of the software product.

It is worth mentioning that our approach does not claim to be unique. Its value is in the fact that we have collected and analyzed aspects of various quality assessment methodologies, tested them in our work on custom projects and compiled a IT product evaluation checklist of requirements sufficient for an adequate and accurate assessment.

Determining software requirements in IT projects is a very important task in the initial phase of project development. Properly defined requirements are a guarantee that the system will meet the requirements of those interested in its development, the so-called stakeholders. For large systems, the number of requirements can be quite significant and the requirements can change, in addition, they are often related. Moreover, there is always the possibility that some of the requirements of stakeholders will not be implemented or incorrectly implemented. That’s why we create our own checklist of requirements for IT products.

The main problem with requirements definition is that it is very difficult to identify the final requirements before the implementation of the software product, as the process of requirements identification is often informal.

In addition, requirements are collected from different sources, which also makes it difficult to interpret them unambiguously and track changes.

Main difficulties in defining IT project requirements

• When determining software requirements, it is necessary to take into account the wishes of a large number of stakeholders, which is typical for such projects;
• A variety of requirements of different types. Each type of requirement should be described with a certain degree of detail;
• Requirements can be complex hierarchical structures that need to be maintained;
• Requirements of various types can be related to each other, therefore, it is necessary to track the traceability of requirements, that is, to establish, change and remove the relationship between them.

These reasons lead to the fact that the requirements change in the course of the project, which, in our opinion, is primarily due to the insufficient degree of their definition at the beginning of the design. Our IT project assessment checklist will make it easier for you to collect customer requirements.

Organization of collection, analysis and evaluation of ideas to collect requirements.

Solving these problems affects the entire project development process, affecting information, communications, and tools. The main problem in this case is the coordination of personnel actions, the organization of cooperation. Many modern companies use a global software supply chain, which includes both various divisions within the company and divisions that are geographically and temporally separated, which greatly complicates their coordination;

Scalability of practical skills.

The informal methods of identifying requirements, which employees often choose when working on small projects, tend to be disappointing when working on larger projects. In addition, projects in which the majority of employees work remotely are very different from projects carried out by a team with a compact arrangement of employees;

The required degree of documentation and the need for tools may vary depending on the size of the project.

Many companies use Word text documents, PowerPoint presentations, Visio diagrams, and Excel spreadsheets as a tooling environment for identifying requirements. The latter can be used to track requirements and where they arise. Such a toolkit can be used in small projects. When performing large or multiple projects, such tools are ineffective;

Integration of information when using such tools Word, PowerPoint, Visio, Excel is limited.

These tools can support embedding and updating of certain information, but they have limited capabilities when working with relational data. The ability to track intermediate results is lost, making it difficult to reuse previously obtained results;

The need to support work with requirements online.

Word tools, PowerPoint Visio, Excel do not support interactive work. When using these tools, you can create repositories for requirements artifacts, use discussion forums, or use email to resolve issues. This still creates scattered information that still needs to be collected and cataloged to get context.

Our software project evaluation checklist consists of several blocks. Let's take a look at each of them.

1. Functional requirements for “successful” scenarios.

Functional requirements are a list of services that the system must perform, and it should be indicated how the system reacts to certain input data, how it behaves in certain situations, etc. In some cases, it is indicated what the system should not do.

Standard forms for specifying functional requirements:

• A description of a function or object.
• Description of the input data and their sources.
• Description of the output with an indication of their destination.
• An indication of what is needed to execute the function.
• If this is a function specification, a description of the preconditions that must be met before the function is called, and a description of the final condition (postconditions) that must be met after the completion of the function is required.
• Description of side effects (if any).

In this part of the software evaluation checklist template we define the software functionality that developers must build to enable users to accomplish their tasks within the business requirements. Sometimes called behavioral requirements, they contain clauses with the traditional "should" : "The system should send an order confirmation to the user by email."

At this point, we figure out what the system should do. Our developers always have questions for customers about functional requirements.

For example, a client describes the expected result: "The user logs into their personal account, selects this and that, and then goes there." Stop! Where will the personal account come from? How do users register? How to change and recover passwords? Do I need to import / connect an existing user base? Finding out “missed”, implicit requirements is the team’s task.

2. Functional requirements for “emergency” scenarios.

When we buy a car, we think about how it will carry us to various interesting places, and not about how we will have to refuel it.

Typically, the customer presents more or less clearly how the solution will work in positive scenarios.

In our experience, a significant part of the work is spent on processing errors related to external factors: unstable communication, incorrect operation of third-party services, insufficient storage for backups and so on.

3. Non-functional requirements

In this part of the software assessment checklist we will describe the characteristics of the system and its environment, not the behavior of the system. It can also provide a list of restrictions imposed on the actions and functions performed by the system. These include time constraints on the system development process, standards, etc.

Non-functional requirements are not directly related to the functions performed by the system. They are related to the integration properties of the system such as reliability, response time, or system size. In addition, non-functional requirements can impose restrictions on the system, such as I / O bandwidth, or the data formats used in the system interface.

Non-functional requirements are based on budget constraints, take into account the organizational capabilities of the developer company, the ability of the developed system to interact with other software and computing systems, as well as external factors such as safety regulations, legislation on the protection of intellectual property, etc.

1. Performance and power

It is advisable to clarify the figures: the number of users, the response rate for a given amount of data and the load on the system, the amount of stored data, resource consumption of the system during normal operation.

Sometimes the client seeks to minimize labor costs at the expense of everything, including asking for a minimum performance. The project makes it to the prototype stage, and the real load of the production level is getting off scale.

In our practice, there was a case when a client ordered a demo prototype with the possibility of quick deployment on their own laptop, and then launched a dozen users into it without consulting with us. The prototype could not provide the simultaneous operation of so many users.

At the very start, it is important to explain to the customer that you cannot take passengers on a paper plane.

2. Sustainability

We find out MTBF expectations (mean time between failures) at normal, critical and supercritical load. Based on this, you can select the most suitable technological stack and determine whether it is necessary to supplement the solution with such a "side" functionality as advanced monitoring, self-healing, auto-scaling.

The analysis is carried out in parallel with points a, c, e, f, g.

3. Reliability and recoverability

Here we take into account the time to restore the system after a fall, the complete destruction of the entire data center and operational data. We discuss the requirements for data backup and system backup.

4. Security

Do not forget to identify the roles of user groups, the scheme of differentiation of access rights, the requirements for encryption of data and communication channels. This item is especially important for projects with personal data that are stored in the clouds.

What should you do if your client works with ‘sensitive’ personal data and decides to transfer the database from its servers to the cloud? At one of the projects, we did the following - we collected backups from the entire network, encrypted them and put them into cloud storages so that no one who had access to one storage could decrypt anything.

5. Scalability

We always discuss the possibilities of scaling and requirements for it, then we specify the time and cost of deploying additional capacities.

6. Extensibility

Scalability and extensibility have two sides to the coin. We can do a project quickly and inexpensively, but with restrictions on the amount of data, speed, power and other indicators. The customer instantly enters the market, receives feedback, pays less for infrastructure. Another option for the development of events when we are working for the future, but there is a risk that the product will not take off soon and the cost will be higher. Competitors are on the alert. They can do everything quickly and cheaply, but without scalability and extensibility.

Who will the customer ultimately choose as a subcontractor? Someone who is able to show that they understand and will be able to meet the needs of the customer, taking into account restrictions, including financial ones.

It may happen that the customer requests a super-universal solution. The data model / architecture / deployment scheme will turn out to be very expensive for development (without a visible return on the investment horizon). It will require too much time for implementation and high costs for maintenance and infrastructure. That is why it is important to maintain a balance at the start.

7. Serviceability

One of our customers from Britain began to work with clients from their time zone. For a long time, the system could be relatively painlessly stopped for maintenance at night. Then they got users in Australia and Germany. The window of opportunity shrank sharply. And when China connected to their project, it finally shut.

It is important to consider the features of the system: some require availability 99.999% of the time, and you can’t stop them for more than 8+ hours a year.

Find out who will install, maintain, and upgrade the product. If these are admins on the part of the customer, it is important to clarify their qualifications and provide for the appropriate level of complexity for monitoring, diagnosing and updating the system. This requirement is related to documentation, which will be discussed below.

8. Ergonomics

We find out the requirements for usability, accessibility, take into account the time it takes for the average user to master the system, the simplicity of typical scripts, and the built-in documentation. If ergonomics is not worked out on the client side, then UX experts do this.

9. Graphic design

At the start of the project evaluation, we find out whether the customer provides a ready-made design or gives it to our team for development.

10. Documentation

If the client’s employees are involved in the installation and configuration of the solution, documented instructions will obviously be required. It is important to consider the level of qualification of operational personnel. The detail of such an instruction will depend on this.

11. Licensed cleanliness

What to do if the customer wants to use a framework for which he does not have a license? Then you could either buy licenses or start looking for alternatives.

Once we were approached by a company in whose brand book there were paid fonts that the client did not buy. We had to explain the legal consequences. As a result, changes to the brand book were made.

12. Compliance with standards and legislation

What can not be ignored in evaluating an IT project? Federal Law on personal data, compulsory licensing of certain types of activities, fiscal accounting, server location and data storage in certain jurisdictions. The most sensitive projects in this regard are related to the storage and security of personal data.

Subject Area Requirements

Subject area requirements characterize the subject area where the system will be used. These requirements may be functional or non-functional. These requirements reflect the conditions in which the software system will be operated. They can be presented in the form of new functional requirements or in the form of constraints on already formulated functional requirements or in the form of instructions on how the system should perform calculations. Failure to meet the requirements of the subject area can lead to system failure.

Product Requirements

Product requirements describe the performance properties of a software product. These are the requirements for system performance, the amount of required memory, reliability (determines the frequency of possible failures in the system), portability of the system to different computer platforms and ease of use.

Organizational Requirements

This block of software product evaluation template reflect the policies and organizational procedures of the customer and the software developer. These include software product development standards, software implementation requirements (i.e., programming language and design methods), output requirements that determine the timing of software product manufacturing, and related documentation.

Integration Requirements

Integration requirements describe a low-level interface for the interaction of a new system with several other systems in the company. The purpose of this document is to justify and formalize the choice of the integration method. The document contains a description of methods and methods of integration with external systems and services.

Application integration is a technological process used to organize the exchange of data between various information systems using the integration tools provided by applications. Integration tools provided by applications include API functions, processing packages, and data export / import.

User interface requirements

The user interface is part of the software system. User interface requirements can be broken down into two groups:

• Requirements for the appearance of the user interface and forms of interaction with the user
• Requirements for accessing the internal functionality of the system using the user interface

The first group includes the following types of requirements:

• Requirements for placing controls on screen forms
• Requirements for the content and design of displayed messages
• Requirements for input formats

The second group includes the following types of requirements:

• Requirements for the response of the system to user input
• Requirements for response time to user commands

Often, at the initial stages of work, stakeholders have only a general concept of how the final product should look and work. We use special information gathering techniques to turn vague ideas into clear assignments.

There is no single technique that can collect absolutely all the requirements for a product. For each stage of detailing needs, one specific technique or a combination of several is suitable.

At the stage of general business requirements, a one-on-one interview with a customer works well. The requirements of a group of stakeholders are easier to clarify in a group discussion during a brainstorming session. A specially organized working meeting - workshop helps to think over detailed new solutions for the system. In our team, we combine these methods to more fully receive the requirements from the customer.

Our it evaluation checklist can be used in whole or in part, depending on the specific request of a potential customer. The most accurate understanding of what the customer wants to create allows to reduce the uncertainty in the project and increase the accuracy of the assessment. The efforts spent on eliminating uncertainty on the part of the client may indirectly demonstrate their readiness for further cooperation. Hope you like our software evaluation checklist example and it will become useful to you in the future.