Keep the index of the generated reports in the database (#307)

[adrpo]

Prerequisite for #307.

The problem

00_history.html, the index of the regression reports of a branch, lives on the
web server next to them, and is the only record of what has already been
reported. all-reports.py reads it back over HTTP and gives up when it cannot:

urlToOpen = "%s/%s/00_history.html" % (historyurl, branch)
try:
  urlContents = urllib.request.urlopen(urlToOpen).read().decode('utf-8')
except:
  print(urlToOpen + " failed to open")
  missing_branches.append(branch)
  continue

It has to give up, because the alternative is worse: starting from an empty
index would publish a history with only today's report in it and overwrite
everything that came before.

The same line means a branch that has never been published reports nothing on
its first run, and nothing on the next one either - the directory and an empty
00_history.html have to be created on the server first.
createInitialHistoryFilesOnRemote() did that over ssh for the branches spelled
out in the Jenkinsfile, and its call has been commented out for a long time, so
in practice it is done by hand. generateSymbolicJacobian is in the list of
branches reported on and has no index on the server to this day.

That is survivable for a fixed list of branches. It is not survivable for #307,
where every pull request tested creates a pr-<N> branch and nobody can prepare
a directory for a pull request that does not exist yet.

The database is the record, the index is a rendering of it

A history table holds one row per report - the two run dates, the file name and
the four counts:

CREATE TABLE IF NOT EXISTS history (
  branch text NOT NULL, date1 bigint NOT NULL, date2 bigint NOT NULL, fname text,
  improved integer, regressions integer, perfimproved integer, perfregressions integer,
  PRIMARY KEY (branch, date1, date2))

• a branch that has no index gets one, created in the workspace and published
  with the reports by the step that already runs at the end of the report stage,
  so nothing has to be prepared on the server;
• an index that is missing, unreadable or has lost entries is rebuilt from the
  table rather than truncated. A report job that cannot reach the web server no
  longer has to choose between skipping the branch and destroying its history;
• what the server has is still read - it is where the 17464 reports generated
  before the table existed are, and they are copied into it the first time each
  branch is reported on - and the two are compared, so an index that has
  drifted from the database is said out loud and rewritten with everything.

The rendering reproduces the published index exactly. Parsing and re-rendering
all 65 indexes on libraries.openmodelica.org returns all 17464 entries byte for
byte; the only difference is a stray blank line in 19 of them, which is dropped
the first time they are rewritten.

The one case with nothing to go on

No rows in the table and no index to read is either a genuinely new branch or a
server that is unwell. Two things have to agree before it is taken for a new
branch, and anything else leaves the branch alone and reports it as missing,
exactly as before:

1. the history root is served - http://libraries.openmodelica.org/branches/history/
   is fetched once per invocation, so a missing index means a missing file and
   not a broken deployment, a wrong --historyurl or something else answering
   for the host;
2. the database holds at most two runs of the branch, so the report about to
   be generated is its first one and none can have been lost.

A branch whose index is already in the workspace, because it was named twice on
the command line or an earlier invocation started it, continues from that copy;
the second occurrence used to refetch the published index and write the same
report twice.

createInitialHistoryFilesOnRemote() and its commented-out call are removed. The
new table holds no results, so clean-dates.py leaves it alone, as it already
has to leave job_claim alone. doc/README.md documents it with the others.

The second commit drops an unused import cgi from all-reports.py and
all-plots.py; the module was removed in Python 3.13 and both scripts fail at
import time there.

Tested

Against a local sqlite database with three runs of master and a local HTTP
server standing in for libraries.openmodelica.org:

case	before	after
no index, root served, first report of the branch	branch skipped, nothing generated	the report and the index are written, the report is stored
index published, database empty (the first run after this lands)	report generated, index appended	its entries are copied into the database, then as before
index published, nothing new	nothing written	unchanged
index published, one new run	only the new pair generated	unchanged, and the report is stored
web server unreachable, database knows the reports	branch skipped	the complete index is rebuilt from the database
published index truncated	the missing reports would be regenerated and the rest lost	it says so and rewrites it with all of them
no index, empty database, branch has more runs than that	branch skipped	branch skipped, and it says why
no index, history root itself 404	branch skipped	branch skipped, and it says why
branch named twice	report written twice	second occurrence continues from the workspace index
branch with no result table	reported as missing	unchanged

Plus the round trip above over the 65 real indexes. The PostgreSQL side is the
same statements with bigint and %s placeholders; it could not be exercised
against the shared database from here.

---

Generated by Claude Code.