3A systems described in the previous article “3A Systems – Definition“ are data processing systems whose development presents the same problems that need to be addressed and the same decisions that must be made. Getting to know them before getting deep into system planning and development can help us avoid serious mistakes and reach our objectives. Below, I have compiled a list of key questions to be answered by anyone who plans to build a 3A system, no matter its scope, complexity, and life span.
Governance
- What organizational structure or person is responsible for the system? Does he have adequate powers and the knowledge necessary for the development (or management), operation, and maintenance of the system? Is there sufficient funding for all the above?
- Who are the real users of the solution? What are their real requirements and demands? Are they able to clearly state what they need?
- What is the planned life span of the solution? Are the technical solution and the project scope in tune with the planned life span? Is there a roadmap for further development of the solution?
- Do we know what benefits the solution will bring individual users? Does the solution have a clear business case? Has funding and staffing for the solution been secured for its planned life span?
- Does the chosen technology for data management and presentation match the volume of data processed, the complexity of the transformations, and the mode of operation? Do we have any experience with the technology or is it new to us?
- Is there a requirement to allow corrections of errors in published results? If so, what will the process be? Who will have permission to correct data?
- Has it been confirmed that the input data exists? Do we know who is responsible for its preparation and delivery?
Data Acquisition
- Will there be an SLA concerning the availability of the input data?
- How will the communication with the party responsible for delivering data be handled?
- Does the stability and robustness of the planned interfaces reflect the scope and expected life span of the solution? How will the source system owners be informed of any changes during the development and operation of the system?
- How will errors in the input data be detected and resolved? Is the verification of input data quality required?
- What types of checks are required – technical parameters such as the existence of data items, ranges of values, formats of strings, user defined validations? What granularity of validation will be supported – for item values, data rows, relations between values, relations between data sets, and business computations using input data?
- What reactions to errors will be supported – to stop processing, discard error rows, permit repairs, or complete processing?
- How will input data errors be recorded and presented – in error tables, via email, via user interface functions?
Data Adoption
- Who, and on what basis, will define the structure of data in the system from the business perspective?
- Who, and on what basis, will define the data transformations? How will the transformations be developed, documented, and tested?
- Where and how will transformations be performed? Does the technical solution reflect the expected volume of data and the life span of the solution? Do the developers know enough about the technology?
- Will the system provide validations of intermediate results during processing? If so, who will define the tests?
- What reactions will the system provide to the errors discovered – write an error record, stop processing, create a corrective record, inform designated people?
- What scheduler will be used? Will processing start automatically or only when initiated by the user? How long will the processing take and how often will it be repeated?
- Will it be necessary to upload any historical data into the system? Does it exist and is it accessible? Are any special processes or interfaces needed to do it?
Data Accessibility
- Will there be an SLA concerning the availability of results?
- How will the communication with end users be handled? To what extent will the end users themselves be able to change the processing or outputs?
- Is it necessary to notify users about the availability of new results from the 3A system? Is it sufficient if only the latest results are made available?
- Will users get the results themselves or will the results have to be sent to them? Are they technically up to the task of getting them themselves?
- Will the system permit manual inputs and corrections during processing? If so, what processes are required for such inputs and corrections? How will such changes be recorded?
- During the transformation process, will users have the opportunity to change the process run? If so, how will the changes be recorded?
- How will users submit their requests for system changes? Will requests for changes in formats or output structure be allowed? Will there be a help desk available to respond to questions and reported discrepancies?
Development
- Where will the source code be stored and how will it be maintained? If processed data is changed, what process will be in place to reflect the changes in the code? Is it necessary to keep more than one version of the solution or its parts operating in parallel?
- Is the system documented in sufficient detail to allow new users to work with it? Is the development documented sufficiently to allow new developers to jump in?
- Does the system contain a number of components with similar functionality large enough to use code generators? Does the system contain functionality suitable for the use of data driven components?
- Are the development process and the technology applied appropriate for the expected volume of changes in inputs or outputs?
- How many development environments will be used in parallel? How many production environments will be established? How will the transfers between environments be done? How will the environments be synchronized?
- Is test data needed, or can real (production) data be used during development?
- If test data is required, will it be built as synthetic data or by anonymizing production data?
Operation
- What will be the mode of operation? Who will run it – the IT department, application owners, developers? Or will it be the users who control the operation?
- On how many environments will the system run? What are the differences between the development, test, acceptance, and production environments?
- How will errors be handled – unavailability of input data, errors in inputs, errors during processing, user confusion? Who will be responsible for reactions? Is an SLA planned? How many levels of support will be set up?
- What are the requirements for monitoring the operation? How many people will be starting, controlling, and monitoring the operation?
- Is it necessary to provide users with a universal way of searching data in the system? Using what tools?
- Is it necessary to provide users with a universal way of searching system logs or error logs? Using what tools?
- Will the application store historical versions of data? If so, where, in the application itself or by exporting the data to another system? Or will it be up to individual users?
If you have reached this point, I hope that these questions will help you plan and define your next 3A system, whether you are a contractor, a developer, or a user. 42 points! It almost seems that the “Deep Thought” computer in the world of the Hitchhiker’s Guide to the Galaxy has not been mistaken in answering the basic question of Life, the Universe.