"Delete unused stylesheet classes" problems with overloaded styles

[wrCisco]

With the new registered QMetaType and the support in EmbeddedPython.cpp, the direct conversion from QList<int> to QVariant now works great, thanks!

As for the @import rule, the problem now is if I write something plainly wrong, like

@import { src: url(cssprova.css); }

the parser doesn't raise an error, and the serialization ends up with something like this:

@import {;

}

[KevinH]

Yes it is not a fully validating parser (yet). Adding checks on specific direct internal state transitions that are not allowed will catch most of those. In other words transitioning from in import directly to in selector or in property would help detect those cases.

But the older CSSInfo did no validation at all, and incorrectly parsed valid css at times!

I will look into it. Thanks!

[KevinH]

Okay, the very latest version of cssparser2 (not yet posted) will now happily and detect errors of using a "{" in the @import, @charset, and @namespace and report them while trying to continue on gracefully without losing any css.

I have also removed a couple of unused functions from CSSUtils.cpp and .h

I will post a cssparser_v2.1.zip later in the week when I get more free time.

Thanks!

Originally Posted by wrCisco

With the new registered QMetaType and the support in EmbeddedPython.cpp, the direct conversion from QList<int> to QVariant now works great, thanks!

As for the @import rule, the problem now is if I write something plainly wrong, like

@import { src: url(cssprova.css); }

the parser doesn't raise an error, and the serialization ends up with something like this:

@import {;

}

[KevinH]

Okay, I finished porting cssparser v2.1 to use Qt.

See the attached: qcssparser.zip

To build it requires Qt: On my machine you would do the following:
export PATH=${PATH}:/Users/kbhend/Qt5129/bin
unzip qcssparser.zip
cd qcssparser
qmake
make

It has a similar main.cpp as cssparser2 to allow you to see the parser output and the serialized result.

This means I have probably finished with updating cssparser anymore as Sigil would need a parser that understands Qt strings and containers with their built in support for unicode.

All future work will focus on qcssparser and trying to incorporate it into Sigil itself if we decide to go that way in the end.

For the record, I have attached to this issue cssparser_v2.1.zip and qcssparser.zip for anyone who might want to play around with either.

[BeckyEbook]

I unpacked 1000 CSS files from EPUB files, ran cssparser on them and it doesn't look bad.
It found quite a few bugs in purchased files, although – luckily – most of it was fine.


I noticed that cssparser ignores the missing last brace:

.kevin {
    color: blue;

This is a bug for me, http://csslint.net/ shows it nicely.

Spoiler:

For the curious:
There were only 6 files with errors, but as many as 526 there were warnings.
No files had the "Information" status.

The error in the files was always the same: duplicate opening brace.
The warnings (over 5,000 in total) concerned, among others:

• common typing errors (aling, pdding, rightt …)
• non-existent attributes (border-size, cellspacing …)
• attributes related to specific browsers or applications (-webkit…, -o…, -moz…, -ms…, -adobe…)
• attributes with prefix -epub… (-epub-hyphenate, -epub-hyphens, -epub-line-break, -epub-ruby-position, -epub-text-align-last, -epub-text-combine, -epub -text-emphasis-style, -epub-writing-mode)
• font-related attributes, e.g. OpenType features (font-feature-settings, font-kerning, font-variant-numeric …)

[KevinH]

CSSProperties.cpp contains the list of all css3 properties this parser groks.

If there are official properties that are currently valid and commonly used in epubs that should be added to that list, please let me know which ones need to be added and I will add them.

As for the missing final } of a selector, I thought, it provided a warning for that and injected its own. I will look into that.

Thanks for all your testing and feedback!

Originally Posted by BeckyEbook

I unpacked 1000 CSS files from EPUB files, ran cssparser on them and it doesn't look bad.
It found quite a few bugs in purchased files, although – luckily – most of it was fine.


I noticed that cssparser ignores the missing last brace:

.kevin {
    color: blue;

This is a bug for me, http://csslint.net/ shows it nicely.

Spoiler:

For the curious:
There were only 6 files with errors, but as many as 526 there were warnings.
No files had the "Information" status.

The error in the files was always the same: duplicate opening brace.
The warnings (over 5,000 in total) concerned, among others:

• common typing errors (aling, pdding, rightt …)
• non-existent attributes (border-size, cellspacing …)
• attributes related to specific browsers or applications (-webkit…, -o…, -moz…, -ms…, -adobe…)
• attributes with prefix -epub… (-epub-hyphenate, -epub-hyphens, -epub-line-break, -epub-ruby-position, -epub-text-align-last, -epub-text-combine, -epub -text-emphasis-style, -epub-writing-mode)
• font-related attributes, e.g. OpenType features (font-feature-settings, font-kerning, font-variant-numeric …)

[KevinH]

Okay I tested cssparser_v2.1 against the following broken css

.kevin {
    color: blue;


.new { color: Teal; background-color: #FFFF00; }
.deleted { color: red; background-color: #FFFF00; text-decoration: line-through; }

And it produced 2 errors:

Error: 5: Unexpected character '.'in property name
Error: 5: Unexpected character '{'in property name

and the following output:

.kevin {
    color:blue;
    newcolor:Teal;
    background-color:#FFFF00;
}

So without a proper closing selector it kept trying to build the current selector list of properties and values and of course then detected the "{" as being an illegal part of a property name.


If the snippet:

.kevin {
    color: blue;

appeared at the very end of the css, it seems to not detect it at all.

I will add a test at the end of the css parsing that will keep track of nest level and generate an error if not properly nested.

[BeckyEbook]

Here is a list of attributes with the prefix "-epub":
http://idpf.org/epub/301/spec/epub-c...haracteristics
http://idpf.org/epub/301/spec/epub-c...l#sec-css-text
and the following paragraphs

In spec 3.2 some of the attributes prefixed with "-epub" were removed:
https://www.w3.org/publishing/epub3/...l#sec-cdoc-css

[KevinH]

Okay, I have kept track of the selector nesting level and if it reaches the end and is not back to zero, this is now detected as an error in my version of cssparser_v2.2.

KevinsiMac:cssparser2 kbhend$ ./release/cssparser/cssparser ~/Desktop/junk.css
Errors: 1
  Error: 5: Unbalanced selector braces in style sheet
Warnings: 0
Information: 0
Pos: 0 Type: SEL_START  Data: .kevin
Pos: 13 Type: PROPERTY  Data: color
Pos: 20 Type: VALUE  Data: blue
.kevin {
    color:blue;

[KevinH]

Okay the final version of qcssparser_v2.2.zip is attached. It now tracks line number as well as file position. It also detects the errors that it missed ala BeckEBook's bug report. I have also added the -epub specific css properties that are valid in the epub 3.2 spec to the qCSSProperties list.

I will also attached the final version of cssparser_v2.2.zip here just for the record.

Sigil master now has the Qt versions of these files as well

[KevinH]

FWIW - I have now modified the old CSSInfo.cpp approach used to identify unused stylesheet classes to walk very tenderly when seeing selectors that use combinators (blank, +, ~, >).

It will basically only look at the first part of the selector (pre combinator) and if that is used at all then assume that its full selector (with combinator) is used.

This will buy us some time to come up with the better solution for Sigil

Right now the qCSSParser should help but without a really good Query library we will be out of luck and testing that and trying to improve it will be my next project.

Even if we get that working on some limited set of queries, if wrCisco's python3lib approach can work, we may be better off using that as it is much more proven.

[KevinH]

Hi All,

I have started work on the gumbo query code and have fixed a few more things in their selector parser code.

So what I need is a large set of css stylesheets that I can parse to extract the selectors from and pass them to the Query parser to stress test it.

If anyone is willing to share a large set of actual css stylesheets with me (just stylesheets not epubs) please post me with a link to "KevinH".

One question for those associated with MR's set of public domain epubs, how fancy are the css stylesheets used in those epubs in general? I am specifically looking for stylesheets that use pseudo elements and pseudo classes or combinators.

Do you think I would I be able to generate a my own good set of stylesheets by randomly downloading a set of public domain epubs from MR and extracting their style sheets? Or would they all just be standard element and .class selectors?


Thanks

[DNSB]

Originally Posted by KevinH

One question for those associated with MR's set of public domain epubs, how fancy are the css stylesheets used in those epubs in general? I am specifically looking for stylesheets that use pseudo elements and pseudo classes or combinators.

Do you think I would I be able to generate a my own good set of stylesheets by randomly downloading a set of public domain epubs from MR and extracting their style sheets? Or would they all just be standard element and .class selectors?


Thanks

Pretty well all of the MR epubs I've downloaded have had simple stylesheets.

[PeterT]

I wonder is some of the stylesheets used by Standard eBooks https://standardebooks.org/ might be of use.

[KevinH]

Excellent suggestion! I did not know that site even existed. I grabbed one epub randomly (the advanced Readium version) and it had 3 css sheets that involved a number of complex selectors.

This will be a great source for tests.

Thank you!

Originally Posted by PeterT

I wonder is some of the stylesheets used by Standard eBooks https://standardebooks.org/ might be of use.