Hugin Panorama Creator Thinks Identically Scanned Grayscale Images Are Colour

[Java Jive]

Regulars may remember some posts of mine from a year or two back
concerning scanning ancient family documents. A cousin of mine has done
a great deal of work in genealogy, creating some pretty large family
trees which I've scanned in sections. By hook or by crook, I've managed
to stitch together all the smaller ones, mostly 3x3, 3x4, or 3x5 scans,
but even on some of those I've seen some weird behaviour when trying to
use Hugin to stitch the sections together. Particularly that some
scans, although originally scanned identically to the others, are
assessed by Hugin to be somehow different from the others.

A couple of days ago, on a previous tree now completed, I was getting a
message that one scan was colour when all the others were greyscale, but
all other imaging software that I loaded it into showed it to be
greyscale. The only thing that was different about this section was
that I noticed I'd begun to move the document fractionally too soon
before the scan had completed, and that consequently a thin slice of the
trailing edge of it was corrupted, but, as it was mostly empty, to save
myself having to set up everything all over again just to rescan that
section, I'd copied the bit that wasn't empty from the overlapping edge
of another successful scan. For some reason, Hugin alone stubbornly
maintained that the result was a colour scan, and in the end I did have
to go through the chore of rescanning the single section so that I could
stitch it.

Now, I'm trying to do a large tree of 4x15 scanned sections - which I
think must be drawn on something like wallpaper backing paper - and,
despite all the scans being done identically as greyscale, Hugin
maintains that the first two sections are greyscale and all the rest are
colour ...

Greyscale section acknowledged by Hugin as greyscale:
www.macfh.co.uk/Temp/01B.png
Greyscale section claimed by Hugin to be colour:
www.macfh.co.uk/Temp/01C.png

Can anyone see what the hell is going on here? What is different about
the first two scans, and why does it think all the others are colour
when no other software that I've tried agrees with it?

[Anton Shepelev]

[alt.os.linux dropped because my comment is OT]

Post by Java Jive
Regulars may remember some posts of mine from a year or
two back concerning scanning ancient family documents.

I am not a regular, but came by your website from a your
post about color-correction (which I like), and man did I
enjoy your Reminiscences!

[Java Jive]

Post by Anton Shepelev
I am not a regular, but came by your website from a your
post about color-correction (which I like), and man did I
enjoy your Reminiscences!

Thanks!

[Andy Burns]

Post by Java Jive
Can anyone see what the hell is going on here?

I've only used Hugin/Panotools/PTgui with full colour images, have you tried
converting e.g. to greyscale TIFF?

[Java Jive]

Post by Andy Burns

Post by Java Jive
Can anyone see what the hell is going on here?

I've only used Hugin/Panotools/PTgui with full colour images, have you
tried converting e.g. to greyscale TIFF?

I don't know how or why it came about, but there does seem to be a
difference between the two sets of images. The first two being seen by
all software as greyscale, the remainder by Windows software as
greyscale, but by GIMP as 256 colour (where presumably the 'colour'
palette was composed entirely of shades of grey).

So I used IrfanView Thumbnail viewer to convert them all first to 16.7m
colour bitmaps, *.bmp, then the bitmaps back to greyscale *.png, and
this seems to have sorted the problem. The new *.png all have loaded
into Hugin.

But it would be nice to know:

- How they came to be scanned differently in the first place?

- Why the Windows software I used to examine them couldn't tell the
difference in format?

[Java Jive]

Post by Java Jive
So I used IrfanView Thumbnail viewer to convert them all first to 16.7m
colour bitmaps, *.bmp, then the bitmaps back to greyscale *.png, and
this seems to have sorted the problem.  The new *.png all have loaded
into Hugin.

But this is the unusable result:

www.macfh.co.uk/Temp/Hugin.png

Rrrrrrrr!

[Andy Burns]

promote them all toc olour?

Post by Java Jive
www.macfh.co.uk/Temp/Hugin.png
Rrrrrrrr!

[Mike]

Post by Java Jive
Can anyone see what the hell is going on here?

FFMPEG :-

$ ffprobe Temp01B.png

Input #0, image2, from 'Temp01B.png':
Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
Stream #0:0: Video: png, gray, 1607x2177, 25 tbr, 25 tbn, 25 tbc

$ ffprobe Temp01C.png

Input #0, image2, from 'Temp01C.png':
Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
Stream #0:0: Video: png, pal8, 1624x2173, 25 tbr, 25 tbn, 25 tbc

Note: "png gray" vs "png pal8" ...

Whereas Imagemagick shows no immediate difference, resolution aside :-

$ identify Temp01B.png
Temp01B.png PNG 1607x2177 1607x2177+0+0 8-bit PseudoClass 256c 2.493MB 0.000u 0:00.000

$ identify Temp01C.png
Temp01C.png PNG 1624x2173 1624x2173+0+0 8-bit PseudoClass 256c 2.761MB 0.000u 0:00.000

Both are 256 colour images. One is 256 shades of grey, the other
is 8-bit, 256-*colour* palette.

Did you EDIT the second image in any way (as in load/save
in anything?) -- I find sometimes apps can decide to save
as colour (by default) when the input was greyscale. Others
are smarter and see grey loaded, so save as grey.

Same problem with e.g. loading an UNCOMPRESSED TIFF (as scanned),
and it getting saved out as COMPRESSED TIFF -- rendering it unloadable in
certain other (less capable) software.

You could force your edit-software to save as grey (there should
be SOME checkbox option?) or clean up after it by forcing colour ones
back to grey with something like :

$ mogrify -type Grayscale myimage.png

Note: which *replaces* the original with the newly "grayed" one, there
is no "output file".

It seems to have co-erced your Temp01c.png into the correct format ...

$ ffprobe Temp01C.png

Input #0, image2, from 'Temp01C.png':
Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
Stream #0:0: Video: png, gray, 1624x2173, 25 tbr, 25 tbn, 25 tbc
^^^^^^^^

[Java Jive]

Post by Mike

Post by Java Jive
Can anyone see what the hell is going on here?

$ mogrify -type Grayscale myimage.png

Yes, thank you.

I found that both Windows apps, IrfanView and PaintShop Pro 8, treat
both genuine grayscale and 8-bit palette with greyscale colours as
'greyscale', so there's nothing to be done there. Linux GIMP can
convert each file manually under Image, Mode, but that's tedious, so
before reading your post I searched on the web for a batch command, and
so discovered ...
mogrify -colorspace Gray *.png
... which has solved the problem, and I'm now in business, though have
yet to see whether I'm going to have to set the control points manually
or automatically.

For some reason or other, probably too many areas looking the same, the
latter tends not to work well with this type of input, and I've been
getting botched results with it. The manual method, though tedious,
tends to work quite well though.

Every now and then, I have a piece of luck and the Windows Image
Composite Editor (ICE) program gets it right first time, which happened
with the last tree, but sadly not with this one.

[Paul]

Post by Java Jive

Post by Mike

Post by Java Jive
Can anyone see what the hell is going on here?

$ mogrify -type Grayscale myimage.png

Yes, thank you.
I found that both Windows apps, IrfanView and PaintShop Pro 8, treat both genuine grayscale and 8-bit palette with greyscale colours as 'greyscale', so there's nothing to be done there.  Linux GIMP can convert each file manually under Image, Mode, but that's tedious, so before reading your post I searched on the web for a batch command, and so discovered ...
    mogrify -colorspace Gray *.png
... which has solved the problem, and I'm now in business, though have yet to see whether I'm going to have to set the control points manually or automatically.
For some reason or other, probably too many areas looking the same, the latter tends not to work well with this type of input, and I've been getting botched results with it.  The manual method, though tedious, tends to work quite well though.
Every now and then, I have a piece of luck and the Windows Image Composite Editor (ICE) program gets it right first time, which happened with the last tree, but sadly not with this one.

Looking at them with a hex editor, one of the images has an obvious
lookup table for the colours. The other does not. The other one
has the four character string "gAMA" as if a gamma curve is being used.
These are structured ahead of an IDAT section. I only examined the
content of each, until I hit IDAT. IDAT is at 0x5D on one image,
and on the other the IDAT is after the lookup table and is at 0x331.

One thing about PNG as a format, is output routines like to try a number
of formats for storage, compare the file size and keep the smallest one.
This notion originated with a piece of software used to reduce PNGs
so they could be served on the web. Not to be outdone, it would seem
some popular library has also taken it upon themselves to do a
version of that reductionism. The difference between them, is the original
concept of PNG reduction, did an inordinate number of versions, and had a long
run time (nobody cared, since this was for a web server and the effort
would "pay for itself"). The library version, who can say how many it
has tried, before picking one.

If you switch to some other image family (someone mentioned TIF), perhaps
the library that handles those, will "stick to the theme you selected"
and not mess around like PNG does. JPEG would not be a good choice, because
of the rounding errors in the colour space (good conversion routines can
rationalize the colour space and return it to normal, but with your
luck so far, this doesn't sound like a good idea :-) ).

Maybe BMP would work, but I don't know enough about the
innards there to comment.

*******

In GIMP, Image : Mode : Grayscale to start.
Then Colours : Threshold and move the slider for a more pleasing result.
This will irritate the hell out of the panorama software though.

Using the threshold, I can keep the text, with only a bit of
noise showing through. There won't be much of anything for the
panorama software to latch onto, but at least the result will be clean.

[Picture]

Loading Image...

Maybe there's a better adaptive filter out there, to
pull signal out of noise. As using Threshold "chews holes"
in the writing which is not good for overall result.

Looking at the image, it almost looks like something
where a different colour of illumination (UV?) might
pick out the writing better. The "ink" looks different
to my eye than the noise in the image.

Maybe something that recognizes hand writing could
fix up the quality of the scan.

Paul

[Jasen Betts]

Post by Java Jive
A couple of days ago, on a previous tree now completed, I was getting a
message that one scan was colour when all the others were greyscale, but
all other imaging software that I loaded it into showed it to be
greyscale. The only thing that was different about this section was
that I noticed I'd begun to move the document fractionally too soon
before the scan had completed, and that consequently a thin slice of the
trailing edge of it was corrupted, but, as it was mostly empty, to save
myself having to set up everything all over again just to rescan that
section, I'd copied the bit that wasn't empty from the overlapping edge
of another successful scan. For some reason, Hugin alone stubbornly
maintained that the result was a colour scan, and in the end I did have
to go through the chore of rescanning the single section so that I could
stitch it.
Now, I'm trying to do a large tree of 4x15 scanned sections - which I
think must be drawn on something like wallpaper backing paper - and,
despite all the scans being done identically as greyscale, Hugin
maintains that the first two sections are greyscale and all the rest are
colour ...
www.macfh.co.uk/Temp/01B.png
www.macfh.co.uk/Temp/01C.png
Can anyone see what the hell is going on here? What is different about
the first two scans, and why does it think all the others are colour
when no other software that I've tried agrees with it?

$ file 01[BC].png
01B.png: PNG image data, 1607 x 2177, 8-bit grayscale, non-interlaced
01C.png: PNG image data, 1624 x 2173, 8-bit colormap, non-interlaced

[Mike Scott]

Post by Java Jive
Regulars may remember some posts of mine from a year or two back
concerning scanning ancient family documents.  A cousin of mine has done
a great deal of work in genealogy, creating some pretty large family
trees which I've scanned in sections.  By hook or by crook, I've managed
to stitch together all the smaller ones, mostly 3x3, 3x4, or 3x5 scans,
but even on some of those I've seen some weird behaviour when trying to
use Hugin to stitch the sections together.  Particularly that some
scans, although originally scanned identically to the others, are
assessed by Hugin to be somehow different from the others.

On a tangent, are you using the right tool for the job? Stitching images
of bits of family tree sounds a lot of effort that might be better
directed towards using a genealogy database program -- 'gramps' comes to
mind.

[Java Jive]

Post by Mike Scott

Post by Java Jive
Regulars may remember some posts of mine from a year or two back
concerning scanning ancient family documents.  A cousin of mine has
done a great deal of work in genealogy, creating some pretty large
family trees which I've scanned in sections.  By hook or by crook,
I've managed to stitch together all the smaller ones, mostly 3x3, 3x4,
or 3x5 scans, but even on some of those I've seen some weird behaviour
when trying to use Hugin to stitch the sections together.
Particularly that some scans, although originally scanned identically
to the others, are assessed by Hugin to be somehow different from the
others.

On a tangent, are you using the right tool for the job? Stitching images
of bits of family tree sounds a lot of effort that might be better
directed towards using a genealogy database program -- 'gramps' comes to
mind.

Oh yes, and thanks for the name which I will note, but that's for the
future. The urgent task at the moment is the preservation of primary
documents that have been the work of many generations before they
deteriorate any further and so that we can still access them even after
handing them over to the care of one of the national document archives.

[dillinger]

Post by Java Jive
Regulars may remember some posts of mine from a year or two back
concerning scanning ancient family documents.  A cousin of mine has done
a great deal of work in genealogy, creating some pretty large family
trees which I've scanned in sections.  By hook or by crook, I've managed
to stitch together all the smaller ones, mostly 3x3, 3x4, or 3x5 scans,
but even on some of those I've seen some weird behaviour when trying to
use Hugin to stitch the sections together.  Particularly that some
scans, although originally scanned identically to the others, are
assessed by Hugin to be somehow different from the others.
A couple of days ago, on a previous tree now completed, I was getting a
message that one scan was colour when all the others were greyscale, but
all other imaging software that I loaded it into showed it to be
greyscale.  The only thing that was different about this section was
that I noticed I'd begun to move the document fractionally too soon
before the scan had completed, and that consequently a thin slice of the
trailing edge of it was corrupted, but, as it was mostly empty, to save
myself having to set up everything all over again just to rescan that
section, I'd copied the bit that wasn't empty from the overlapping edge
of another successful scan.  For some reason, Hugin alone stubbornly
maintained that the result was a colour scan, and in the end I did have
to go through the chore of rescanning the single section so that I could
stitch it.
Now, I'm trying to do a large tree of 4x15 scanned sections  -  which I
think must be drawn on something like wallpaper backing paper  -  and,
despite all the scans being done identically as greyscale, Hugin
maintains that the first two sections are greyscale and all the rest are
colour ...
    www.macfh.co.uk/Temp/01B.png
    www.macfh.co.uk/Temp/01C.png
Can anyone see what the hell is going on here?  What is different about
the first two scans, and why does it think all the others are colour
when no other software that I've tried agrees with it?

Did your scanning software save everything as grayscale?
Does it perhaps have some autodetect feature which detected some
documents as grayscale