Data Architecture Considerations
Organisations transform, and talking about digital transformation - data becomes one of the core aspect of management. In today’s world, data is of prime importance. Even when simple applications tend to grow, the data complexity quickly escalates and thus it becomes important to give it a structure with apt understanding at the very beginning. Data model helps you as any stakeholder understand the data landscape with respect to the business, applications, physical storage, flows, relationships, archival and currency. At the very least it provides the high level views of how the data is scattered across different worlds.
In terms of Enterprise Architecture, we should take this job seriously while defining the baseline and target data architectures. The architecture definition document should have below considerations while building and documenting data architectures and related principles.
Probably the most important of all. Enough is said about data security already.
In today’s world government regulations change fast and we have seen enough sensitiviy with respect to data protection. A study should be conducted to learn about data protection laws and regulations based on the region where the data is going to be stored, people who deal with the data, flow of data between various parties and processes etc. Analysis report listing out the full proof constraints should be made ready before we even dive into defining data models.
When defining data models at various levels, it is important to know where your organisation is heading. Such understanding enables us to modularize data models in order to be as ready as possible for the change that will be ahead of us in the future. Whenever an organisation is ready for a change/transformation, current data model that is being developed should support it in a way that it has minimal impact and reduces the time to reach target definition.
Data is consumed by various functions within an organisation. Most of the times, confidential data used by various functions reside on same “database” - or let’s say same set of hardware. Knowing and publishing the facts about who should access what enables the data architect to define appropriate access controls at various levels. This should also have a pointers towards interfacing requirements within various applications for accessing the data. If required, measures should be taken to sign a contract when a new person is being onboarded on a critical function.
We all know natural calamities happen and they are often not in our control. But taking care of our belongings can be done by having a solid plan in the form of Disaster Recovery. Data architecture document should have a plan in place to switch over to another database with least amount of data loss and downtime. This plan should ensure consistency in terms of entire end to end enterprise architecture.
Data flow diagrams are very critical in making sure entire Architecture Board understand how the data is going to flow within functions, processes, regions, servers, applications and users. A correct data flow diagram should trigger thoughts around Technology Architecture with respect to network design, Encryption requirements, data center design, provision of various segments of network bandwidth, implementation of firewalls etc. Data flow diagrams should be well thought of in a way that makes sense and is modifiable with minimum amount of disruption.
Data architect should be well aware of the storage requirements of various forms of data. A separate section should be dedicated indicating the storage requirement over a period of time with respect to data centers or servers - whichever is more sensible and needs to be indicated. Additionally, it should also define policies for data archival and retrieval.
Data model helps the board in the process of development of a complete data model. It should start with definition of conceptual data model, followed by logical and then physical which describes in details, the final data model which is currently set up as well as the target data model which enables the desired state. Details about the attributes and their data types should be represented in such a way that gives the idea about the limits of memory sizes that can be occupied by a particular attribute related to an entity. This data model should also define cardinality between various entities being defined in order to understand the relationships. Usage of Entity-Relationship diagrams is mandated as it forms the common language for all the stakeholders to understand and be on same page.