Top 2021 CSS Frameworks Report, Part 2: Validation

This is the second part of the Top CSS Frameworks of 2021 report. Here’s the first part of the report about the file sizes.

In this report, I have audited top CSS frameworks (according to the State of CSS report) for errors and warnings. Here’s the complete list of audited frameworks:

• Ant Design
• Bootstrap
• Bulma
• Foundation
• Halfmoon
• Materialize
• Primer CSS
• Pure CSS
• Semantic UI
• Tachyons
• TailwindCSS
• UIkit

⚠️ It is worth noting that I had to use the previous version of the TailwindCSS since I could not fully download the latest one.

The Tooling

For this report, I used the following tools:

• css-validator

Using this tool, I validated CSS code against the W3C CSS Validation Service. In addition, all errors and warnings are validated against the css3svg profile.

⚠️ Any errors in the report that might be caused by invalid software are not deliberate and should be considered involuntary.

The Report

Here are the final numbers:

Errors

Let’s start with frameworks that did a good job and that don’t have any errors in code:

• Bootstrap
• Pure CSS
• Tachyons
• TailwindCSS
• UIkit

When using a framework, you expect to get the code without errors, but only 7 out of 12 CSS frameworks deliver error-free code.

The rest of the frameworks have at least one error. Two frameworks with the most errors are Semantic UI and Primer CSS. Semantic UI has around 100 errors, of which 78 are related to invalid values.

Primer CSS has no less than 4025 errors! The majority of these errors are related to a strange snippet that looks like JavaScript code that is included in the code a few thousand times: (obj) => get_1.default(obj, path). Since I audited an earlier version of the framework, 19.2.0, in January, I checked again with the latest version, 19.8.2. To my surprise, there were even more errors, shocking 5268 errors.

The average number of errors is 346, and the median is 13.

Error types

These are the discovered types of errors:

• parser error - general error,
• error in value - when there are too many values provided,
• negative value is not allowed - when the provided value cannot be a negative number,
• property doesn't exist in “CSS Level 3 + SVG” profile - when the provided property doesn’t exist in the current validation profile,
• property doesn't exist - when the provided property doesn’t exist in any validation profile,
• is not a valid RGB color - when the provided value is not 3 or 6 hexadecimal number,
• too many values or values are not recognized - when there are too many values provided or when provided values are not recognized for the given property,
• is not a value - when the provided value is not a proper value for this property,
• only “0” can be a “unit” - when the provided value is missing a unit.

⚠️ Note that this is my interpretation of error meanings. Unfortunately, I couldn’t find the official list of all errors described in detail.

According to the W3C CSS Validation Service translation keys, many different errors could occur (more than 80), and the parser found only nine different ones when validating these frameworks. Here’s the list of all error types by CSS framework:

Most of the errors are related to values — either the values are out of range (negative values), zero value has a unit when it shouldn’t, or the value is invalid. The other three error types are a general error when the parser cannot read the line and an error when a property doesn’t exist. It is worth noting that the css-validator tool reported the following CSS properties as non-existing:

• font-smoothing
• tap-highlight-color
• font-variant
• text-decoration
• text-transform

The font-smoothing property actually doesn’t exist, but tap-highlight-color does, but it is a non-standard property. The rest of the properties exist but are not supported in the “CSS Level 3 + SVG” profile. I think it has to do with the SVG part since these are supported in other profiles.

Warnings

Every validated CSS framework has at least a few dozens of warnings, but there are only two frameworks with less than a hundred warnings: Pure CSS and Tachyons. On the other hand, there are two frameworks with more than a thousand warnings: Halfmoon and Semantic UI. The rest are between these two values with an average of 507 warnings.

Warning types

The parser found 14 different warnings. Here’s the list with explanations:

• not-needed at-rule - when the provided at-rule is prefixed and might be obsolete or outdated,
• CSS hack - when the provided declaration is considered as a hack,
• CSS variable - when a CSS variable is used,
• Deprecated value - when the provided value is deprecated,
• Deprecated property - when the provided property is deprecated,
• no declaration - when there is no declarations in the rule,
• Property doesn't exist for media - when the provided property for the given media cannot work,
• Out of range value - when the provided value is out of range for the given property,
• Invalid separator - when the provided separator between values is not valid,
• Unsupported import - when an import is used,
• Unofficial value - when the provided value is not officially specified but is supported in multiple browsers,
• not-needed pseudo-class - when the provided pseudo-class is prefixed and might be obsolete or outdated,
• not-needed pseudo-element - when the provided pseudo-element is prefixed and might be obsolete or outdated,
• not-needed property - when the provided property is prefixed and might be obsolete or outdated.

⚠️ Note that this is my interpretation of warning meanings. Unfortunately, I couldn’t find the official list of all warnings described in detail.

Although most of these warnings could be ignored, I found them really insightful. For example, I noticed that every framework uses prefixed properties, pseudo-elements, and pseudo-classes. I also discovered that only five frameworks use CSS variables, and three use CSS hacks. And did you know that Semantic UI imports Lato font from Google Fonts? I didn’t, and I couldn’t figure out why.

I also noticed that the parser complained about the clip property for the deprecated value warning. I found out that the property is indeed deprecated, according to caniuse. The parser also complained about the auto value for the pointer-events property for the unofficial value warning. I didn’t know that the auto value is not officially supported. Finally, I noticed that the parser complained about the transition property in the @media print rule. It makes sense not to include transitions when printing.

The Conclusion

I must admit I didn’t use the css-validator service very often. It was too time-consuming to go through all errors and warnings and try to fix them. After all, the code worked thanks to the forgiving nature of the CSS.

But what would happen if the browser breaks when it finds an error in CSS? Imagine how many sites would stop working. We complain about CSS all the time, but CSS is too kind to us. It lets us make mistakes unpunished.

Now, go ahead and test your site against the W3C CSS Validation Service and see how many errors and warnings do you have. I was surprised that my personal site has zero errors and warnings. And if you are wondering if this site has any errors, it doesn’t.

#RespectCSS

Related resources

• Report: Top 2021 CSS Frameworks Report, Part 1: The CSS File Sizes