Database_normalisation

Database normalization

Database normalization

Reduction of data redundancy


Database normalization is the process of structuring a relational database in accordance with a series of so-called normal forms in order to reduce data redundancy and improve data integrity. It was first proposed by British computer scientist Edgar F. Codd as part of his relational model.

Normalization entails organizing the columns (attributes) and tables (relations) of a database to ensure that their dependencies are properly enforced by database integrity constraints. It is accomplished by applying some formal rules either by a process of synthesis (creating a new database design) or decomposition (improving an existing database design).

Objectives

A basic objective of the first normal form defined by Codd in 1970 was to permit data to be queried and manipulated using a "universal data sub-language" grounded in first-order logic.[1] An example of such a language is SQL, though it is one that Codd regarded as seriously flawed.[2]

The objectives of normalisation beyond 1NF (first normal form) were stated by Codd as:

  1. To free the collection of relations from undesirable insertion, update and deletion dependencies.
  2. To reduce the need for restructuring the collection of relations, as new types of data are introduced, and thus increase the life span of application programs.
  3. To make the relational model more informative to users.
  4. To make the collection of relations neutral to the query statistics, where these statistics are liable to change as time goes by.
E.F. Codd, "Further Normalisation of the Data Base Relational Model"[3]
An insertion anomaly. Until the new faculty member, Dr. Newsome, is assigned to teach at least one course, their details cannot be recorded.
An update anomaly. Employee 519 is shown as having different addresses on different records.
A deletion anomaly. All information about Dr. Giddens is lost if they temporarily cease to be assigned to any courses.

When an attempt is made to modify (update, insert into, or delete from) a relation, the following undesirable side effects may arise in relations that have not been sufficiently normalized:

Insertion anomaly
There are circumstances in which certain facts cannot be recorded at all. For example, each record in a "Faculty and Their Courses" relation might contain a Faculty ID, Faculty Name, Faculty Hire Date, and Course Code. Therefore, the details of any faculty member who teaches at least one course can be recorded, but a newly hired faculty member who has not yet been assigned to teach any courses cannot be recorded, except by setting the Course Code to null.
Update anomaly
The same information can be expressed on multiple rows; therefore updates to the relation may result in logical inconsistencies. For example, each record in an "Employees' Skills" relation might contain an Employee ID, Employee Address, and Skill; thus a change of address for a particular employee may need to be applied to multiple records (one for each skill). If the update is only partially successful – the employee's address is updated on some records but not others – then the relation is left in an inconsistent state. Specifically, the relation provides conflicting answers to the question of what this particular employee's address is.
Deletion anomaly
Under certain circumstances, the deletion of data representing certain facts necessitates the deletion of data representing completely different facts. The "Faculty and Their Courses" relation described in the previous example suffers from this type of anomaly, for if a faculty member temporarily ceases to be assigned to any courses, the last of the records on which that faculty member appears must be deleted, effectively also deleting the faculty member, unless the Course Code field is set to null.

Minimize redesign when extending the database structure

A fully normalized database allows its structure to be extended to accommodate new types of data without changing existing structure too much. As a result, applications interacting with the database are minimally affected.

Normalized relations, and the relationship between one normalized relation and another, mirror real-world concepts and their interrelationships.

Normal forms

Codd introduced the concept of normalization and what is now known as the first normal form (1NF) in 1970.[4] Codd went on to define the second normal form (2NF) and third normal form (3NF) in 1971,[5] and Codd and Raymond F. Boyce defined the Boyce–Codd normal form (BCNF) in 1974.[6]

Informally, a relational database relation is often described as "normalized" if it meets third normal form.[7] Most 3NF relations are free of insertion, updation, and deletion anomalies.

The normal forms (from least normalized to most normalized) are:

More information Constraint (informal description in parentheses), UNF (1970) ...

Example of a step-by-step normalization

Normalization is a database design technique, which is used to design a relational database table up to higher normal form.[9] The process is progressive, and a higher level of database normalization cannot be achieved unless the previous levels have been satisfied.[10]

That means that, having data in unnormalized form (the least normalized) and aiming to achieve the highest level of normalization, the first step would be to ensure compliance to first normal form, the second step would be to ensure second normal form is satisfied, and so forth in order mentioned above, until the data conform to sixth normal form.

However, it is worth noting that normal forms beyond 4NF are mainly of academic interest, as the problems they exist to solve rarely appear in practice.[11]

The data in the following example were intentionally designed to contradict most of the normal forms. In practice it is often possible to skip some of the normalization steps because the data is already normalized to some extent. Fixing a violation of one normal form also often fixes a violation of a higher normal form. In the example, one table has been chosen for normalization at each step, meaning that at the end, some tables might not be sufficiently normalized.

Initial data

Let a database table exist with the following structure:[10]

More information Title, Author ...

For this example it is assumed that each book has only one author.

A table that conforms to the relational model has a primary key which uniquely identifies a row. Two books could have the same title, but an ISBN uniquely identifies a book, so it can be used as the primary key:

More information ISBN, Title ...

Satisfying 1NF

In the first normal form each field contains a single value. A field may not contain a set of values or a nested record.

Subject contains a set of subject values, meaning it does not comply.

To solve the problem, the subjects are extracted into a separate Subject table:[10]

More information ISBN, Title ...
More information ISBN, Subject name ...

In Subject, ISBN is a foreign key: It refers to the primary key in Book, and makes the relationship between these two tables explicit.

Instead of one table in unnormalized form, there are now two tables conforming to the 1NF.

Satisfying 2NF

The Book table below has a composite key of {Title, Format} (indicated by the underlining), which will not satisfy 2NF if some subset of that key is a determinant. At this point in our design the key is not finalised as the primary key, so it is called a candidate key. Consider the following table:

More information Title, Format ...

All of the attributes that are not part of the candidate key depend on Title, but only Price also depends on Format. To conform to 2NF and remove duplicates, every non-candidate-key attribute must depend on the whole candidate key, not just part of it.

To normalize this table, make {Title} a (simple) candidate key (the primary key) so that every non-candidate-key attribute depends on the whole candidate key, and remove Price into a separate table so that its dependency on Format can be preserved:

More information Title, Author ...
More information Title, Format ...

Now, both the Book and Price tables conform to 2NF.

Satisfying 3NF

The Book table still has a transitive functional dependency ({Author Nationality} is dependent on {Author}, which is dependent on {Title}). Similar violations exist for publisher ({Publisher Country} is dependent on {Publisher}, which is dependent on {Title}) and for genre ({Genre Name} is dependent on {Genre ID}, which is dependent on {Title}). Hence, the Book table is not in 3NF. To make it in 3NF, let's use the following table structure, thereby eliminating the transitive functional dependencies by placing {Author Nationality}, {Publisher Country}, and {Genre Name} in their own respective tables:

More information Title, Author ...
More information Title, Format ...
More information Author, Author Nationality ...
More information Publisher, Country ...
More information Genre ID, Name ...

Satisfying EKNF

The elementary key normal form (EKNF) falls strictly between 3NF and BCNF and is not much discussed in the literature. It is intended "to capture the salient qualities of both 3NF and BCNF" while avoiding the problems of both (namely, that 3NF is "too forgiving" and BCNF is "prone to computational complexity"). Since it is rarely mentioned in literature, it is not included in this example.

Satisfying 4NF

Assume the database is owned by a book retailer franchise that has several franchisees that own shops in different locations. And therefore the retailer decided to add a table that contains data about availability of the books at different locations:

More information Franchisee ID, Title ...

As this table structure consists of a compound primary key, it doesn't contain any non-key attributes and it's already in BCNF (and therefore also satisfies all the previous normal forms). However, assuming that all available books are offered in each area, the Title is not unambiguously bound to a certain Location and therefore the table doesn't satisfy 4NF.

That means that, to satisfy the fourth normal form, this table needs to be decomposed as well:

More information Franchisee ID, Title ...

Now, every record is unambiguously identified by a superkey, therefore 4NF is satisfied.

Satisfying ETNF

Suppose the franchisees can also order books from different suppliers. Let the relation also be subject to the following constraint:

  • If a certain supplier supplies a certain title
  • and the title is supplied to the franchisee
  • and the franchisee is being supplied by the supplier,
  • then the supplier supplies the title to the franchisee.[12]
More information Supplier ID, Title ...

This table is in 4NF, but the Supplier ID is equal to the join of its projections: {{Supplier ID, Title}, {Title, Franchisee ID}, {Franchisee ID, Supplier ID}}. No component of that join dependency is a superkey (the sole superkey being the entire heading), so the table does not satisfy the ETNF and can be further decomposed:[12]

More information Supplier ID, Title ...

The decomposition produces ETNF compliance.

Satisfying 5NF

To spot a table not satisfying the 5NF, it is usually necessary to examine the data thoroughly. Suppose the table from 4NF example with a little modification in data and let's examine if it satisfies 5NF:

More information Franchisee ID, Title ...

Decomposing this table lowers redundancies, resulting in the following two tables:

More information Franchisee ID, Title ...

The query joining these tables would return the following data:

More information Franchisee ID, Title ...

The JOIN returns three more rows than it should; adding another table to clarify the relation results in three separate tables:

More information Franchisee ID, Title ...

What will the JOIN return now? It actually is not possible to join these three tables. That means it wasn't possible to decompose the Franchisee - Book - Location without data loss, therefore the table already satisfies 5NF.

C.J. Date has argued that only a database in 5NF is truly "normalized".[13]

Satisfying DKNF

Let's have a look at the Book table from previous examples and see if it satisfies the domain-key normal form:

More information Title, Pages ...

Logically, Thickness is determined by number of pages. That means it depends on Pages which is not a key. Let's set an example convention saying a book up to 350 pages is considered "slim" and a book over 350 pages is considered "thick".

This convention is technically a constraint but it is neither a domain constraint nor a key constraint; therefore we cannot rely on domain constraints and key constraints to keep the data integrity.

In other words – nothing prevents us from putting, for example, "Thick" for a book with only 50 pages – and this makes the table violate DKNF.

To solve this, a table holding enumeration that defines the Thickness is created, and that column is removed from the original table:

More information Thickness, Min pages ...

That way, the domain integrity violation has been eliminated, and the table is in DKNF.

Satisfying 6NF

A simple and intuitive definition of the sixth normal form is that "a table is in 6NF when the row contains the Primary Key, and at most one other attribute".[14]

That means, for example, the Publisher table designed while creating the 1NF:

More information Publisher ID, Name ...

needs to be further decomposed into two tables:

More information Publisher ID, Name ...

The obvious drawback of 6NF is the proliferation of tables required to represent the information on a single entity. If a table in 5NF has one primary key column and N attributes, representing the same information in 6NF will require N tables; multi-field updates to a single conceptual record will require updates to multiple tables; and inserts and deletes will similarly require operations across multiple tables. For this reason, in databases intended to serve online transaction processing (OLTP) needs, 6NF should not be used.

However, in data warehouses, which do not permit interactive updates and which are specialized for fast query on large data volumes, certain DBMSs use an internal 6NF representation – known as a columnar data store. In situations where the number of unique values of a column is far less than the number of rows in the table, column-oriented storage allow significant savings in space through data compression. Columnar storage also allows fast execution of range queries (e.g., show all records where a particular column is between X and Y, or less than X.)

In all these cases, however, the database designer does not have to perform 6NF normalization manually by creating separate tables. Some DBMSs that are specialized for warehousing, such as Sybase IQ, use columnar storage by default, but the designer still sees only a single multi-column table. Other DBMSs, such as Microsoft SQL Server 2012 and later, let you specify a "columnstore index" for a particular table.[15]

See also


Notes and references

  1. "The adoption of a relational model of data ... permits the development of a universal data sub-language based on an applied predicate calculus. A first-order predicate calculus suffices if the collection of relations is in normal form. Such a language would provide a yardstick of linguistic power for all other proposed data languages, and would itself be a strong candidate for embedding (with appropriate syntactic modification) in a variety of host languages (programming, command- or problem-oriented)." Codd, "A Relational Model of Data for Large Shared Data Banks" Archived June 12, 2007, at the Wayback Machine, p. 381
  2. Codd, E.F. Chapter 23, "Serious Flaws in SQL", in The Relational Model for Database Management: Version 2. Addison-Wesley (1990), pp. 371–389
  3. Codd, E.F. "Further Normalisation of the Data Base Relational Model", p. 34
  4. Codd, E. F. (June 1970). "A Relational Model of Data for Large Shared Data Banks". Communications of the ACM. 13 (6): 377–387. doi:10.1145/362384.362685. S2CID 207549016.
  5. Codd, E. F. "Further Normalization of the Data Base Relational Model". (Presented at Courant Computer Science Symposia Series 6, "Data Base Systems", New York City, May 24–25, 1971.) IBM Research Report RJ909 (August 31, 1971). Republished in Randall J. Rustin (ed.), Data Base Systems: Courant Computer Science Symposia Series 6. Prentice-Hall, 1972.
  6. Codd, E. F. "Recent Investigations into Relational Data Base Systems". IBM Research Report RJ1385 (April 23, 1974). Republished in Proc. 1974 Congress (Stockholm, Sweden, 1974), N.Y.: North-Holland (1974).
  7. Date, C. J. (1999). An Introduction to Database Systems. Addison-Wesley. p. 290.
  8. Darwen, Hugh; Date, C. J.; Fagin, Ronald (2012). "A Normal Form for Preventing Redundant Tuples in Relational Databases" (PDF). Proceedings of the 15th International Conference on Database Theory. EDBT/ICDT 2012 Joint Conference. ACM International Conference Proceeding Series. Association for Computing Machinery. p. 114. doi:10.1145/2274576.2274589. ISBN 978-1-4503-0791-8. OCLC 802369023. Archived (PDF) from the original on March 6, 2016. Retrieved May 22, 2018.
  9. Kumar, Kunal; Azad, S. K. (October 2017). "Database normalization design pattern". 2017 4th IEEE Uttar Pradesh Section International Conference on Electrical, Computer and Electronics (UPCON). IEEE. pp. 318–322. doi:10.1109/upcon.2017.8251067. ISBN 9781538630044. S2CID 24491594.
  10. "Database normalization in MySQL: Four quick and easy steps". ComputerWeekly.com. Archived from the original on August 30, 2017. Retrieved March 23, 2021.
  11. "Database Normalization: 5th Normal Form and Beyond". MariaDB KnowledgeBase. Retrieved January 23, 2019.
  12. Date, C. J. (December 21, 2015). The New Relational Database Dictionary: Terms, Concepts, and Examples. "O'Reilly Media, Inc.". p. 138. ISBN 9781491951699.
  13. Date, C. J. (December 21, 2015). The New Relational Database Dictionary: Terms, Concepts, and Examples. "O'Reilly Media, Inc.". p. 163. ISBN 9781491951699.
  14. "normalization - Would like to Understand 6NF with an Example". Stack Overflow. Retrieved January 23, 2019.
  15. Microsoft Corporation. Columnstore Indexes: Overview. https://docs.microsoft.com/en-us/sql/relational-databases/indexes/columnstore-indexes-overview . Accessed March 23, 2020.

Further reading


Share this article:

This article uses material from the Wikipedia article Database_normalisation, and is written by contributors. Text is available under a CC BY-SA 4.0 International License; additional terms may apply. Images, videos and audio are available under their respective licenses.