Wednesday, 2 December 2015

As a friend of mine is member of the DIY bookscanner community and was so generous to lend me his bookscanner, I will document here, how I got up and running the software that is neede to scan books with his scanner.

Installation

Spreads

Homepage: http://spreads.readthedocs.org/en/latest/
Sourcecode: https://github.com/DIYBookScanner/spreads

  1. Install the system dependencies:

    $ sudo apt-get install python2.7-dev python-pip build-essential pkg-config libffi-dev libturbojpeg-dev libmagickwand-dev python-cffi
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    E: Unable to locate package libturbojpeg-dev

    change it to:

    $ sudo apt-get install python2.7-dev python-pip build-essential pkg-config libffi-dev libjpeg-turbo8-dev libmagickwand-dev python-cffi
  2. If you want to use a CHDK device:

    $ sudo apt-get install liblua5.2-dev libusb-dev$ sudo pip install lupa --install-option="--no-luajit"$ sudo pip install chdkptp.py
    CHDK? This means https://de.wikipedia.org/wiki/Canon_Hacker_Development_Kit for compact Canon cameras. In my case I have two Nikon D5300. So I skipped this step.
  3. If you want to use a libgphoto2-supported device (Yes!):

    $ sudo apt-get install libgphoto2-dev$ sudo pip install gphoto2-cffi
    ...
    ImportError: No module named enum

    ----------------------------------------
    Cleaning up...
    Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/gphoto2-cffi
    Storing debug log for failure in /home/ralf/.pip/pip.log

    Fixed it by issuing command:

    $ sudo pip install enum34  Downloading/unpacking enum34
      Downloading enum34-1.1.1.tar.gz (46kB): 46kB downloaded
      Running setup.py (path:/tmp/pip_build_root/enum34/setup.py) egg_info for package enum34
       
    Installing collected packages: enum34
      Running setup.py install for enum34
       
    Successfully installed enum34
    Cleaning up...

    $ sudo pip install gphoto2-cffi
    ...
    Successfully installed gphoto2-cffi
    Cleaning up...

  4. Install Python dependencies:

    $ sudo pip install jpegtran-cffi Flask requests zipstream tornado Wand
    $ sudo pip install http://buildbot.diybookscanner.org/nightly/spreads-latest.tar.gz
    ...
        Installing spread script to /usr/local/bin
    Successfully installed PyYAML futures blinker roman psutil isbnlib pathlib spreads
    Cleaning up...
  5. If you want to use the GUI (Why not?):

    $ sudo apt-get install python-pyside
  6. If you want to use djvu functionality (Not really, but you never know...):

    $ sudo apt-get install djvubind
    Reading package lists... Done
    Building dependency tree      
    Reading state information... Done
    E: Unable to locate package djvubind


    As there was no fast solution, I skipped this installation...

Configuration

$ spread configure
Please select a device driver from the following list:
  [0]: Keep current (None)
  [1]: gphoto2camera
  [2]: chdkcamera
  [3]: dummy
  [4]: None
Select a driver: 1


Please select your desired plugins from the following list:
    1: autorotate
    2: djvubind
    3: gui
    4: hidtrigger
    5: intervaltrigger
    6: pdfbeads
    7: scantailor
    8: tesseract
    9: web
Select a plugin (or hit enter to finish):


You can select one plugin after the other as long you enter a number. When you are finished in selecting plugins just hit enter. I chose 1, 7 and 9:

Select a plugin (or hit enter to finish): 9
  x 1: autorotate
    2: djvubind
    3: gui
    4: hidtrigger
    5: intervaltrigger
    6: pdfbeads
  x 7: scantailor
    8: tesseract
  x 9: web
Select a plugin (or hit enter to finish):


spreads encountered an error:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/spreads/main.py", line 321, in main
    run()
  File "/usr/local/lib/python2.7/dist-packages/spreads/main.py", line 308, in run
    args.subcommand(config)
  File "/usr/local/lib/python2.7/dist-packages/spreads/cli.py", line 210, in configure
    _setup_processing_pipeline(config)
  File "/usr/local/lib/python2.7/dist-packages/spreads/cli.py", line 141, in _setup_processing_pipeline
    exts = [name for name, cls in plugin.get_plugins(*config["plugins"].get())
  File "/usr/local/lib/python2.7/dist-packages/spreads/plugin.py", line 425, in get_plugins
    "extension '{0}':\n{1}".format(err.message, name))
ExtensionException: Error while locating external application dependency for extension 'Could not find executable `scantailor-cli`. Please install the appropriate package(s)!':
scantailor


Ok, that's clear: if there is no scantailor installed, install it:

$ sudo apt-get install scantailor
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following NEW packages will be installed:
  scantailor
0 upgraded, 1 newly installed, 0 to remove and 225 not upgraded.
Need to get 2.056 kB of archives.
After this operation, 4.975 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ trusty/universe scantailor amd64 0.9.11.1-0ubuntu2 [2.056 kB]
Fetched 2.056 kB in 4s (491 kB/s)      
Selecting previously unselected package scantailor.
(Reading database ... 218232 files and directories currently installed.)
Preparing to unpack .../scantailor_0.9.11.1-0ubuntu2_amd64.deb ...
Unpacking scantailor (0.9.11.1-0ubuntu2) ...
Processing triggers for shared-mime-info (1.2-0ubuntu3) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Processing triggers for mime-support (3.54ubuntu1.1) ...
Processing triggers for desktop-file-utils (0.22-1ubuntu1) ...
Setting up scantailor (0.9.11.1-0ubuntu2) ...


Now run configuration again:

$ spread configure...
Select a plugin (or hit enter to finish):
The following postprocessing plugins were detected:
 - autorotate
 - scantailor
Please enter the extensions in the order that they should be invoked, separated by commas or hit enter to keep the current order:

Do you want to configure the target_page of your devices?
(Required for shooting with two devices) [y/N]: y

Setting target page on cameras
Please connect and turn on the device labeled 'odd'
Press any key when ready.


Now connect the USB cable from the USB-hub (to which the cameras connected) into your computer and turn the camera for "odd" page numbers on. It is helpful to place a book into the scanner and open pages with numbered pages to see which page is numbered "odd". Turn on the camera targeting this page. Press any key.

spreads encountered an error:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/spreads/main.py", line 321, in main
    run()
  File "/usr/local/lib/python2.7/dist-packages/spreads/main.py", line 308, in run
    args.subcommand(config)
  File "/usr/local/lib/python2.7/dist-packages/spreads/cli.py", line 232, in configure
    _set_device_target_page(config, target_page)
  File "/usr/local/lib/python2.7/dist-packages/spreads/cli.py", line 177, in _set_device_target_page
    devs = plugin.get_devices(config, force_reload=True)
  File "/usr/local/lib/python2.7/dist-packages/spreads/plugin.py", line 480, in get_devices
    devices = list(driver.yield_devices(config['device']))
  File "/usr/local/lib/python2.7/dist-packages/spreadsplug/dev/gphoto2camera.py", line 48, in yield_devices
    yield cls(config, cam)
  File "/usr/local/lib/python2.7/dist-packages/spreadsplug/dev/gphoto2camera.py", line 63, in __init__
    self._serial_number = unicode(self._camera.status.serialnumber)
  File "/usr/local/lib/python2.7/dist-packages/gphoto2/gphoto2.py", line 577, in status
    config = self._get_config()
  File "/usr/local/lib/python2.7/dist-packages/gphoto2/gphoto2.py", line 89, in wrapped
    cam, ctx = self._cam, self._ctx
  File "/usr/local/lib/python2.7/dist-packages/gphoto2/gphoto2.py", line 767, in _cam
    lib.gp_camera_init(self.__cam, self._ctx)
  File "/usr/local/lib/python2.7/dist-packages/gphoto2/backend.py", line 204, in <lambda>
    return lambda *a, **kw: self._check_error(val(*a, **kw))
  File "/usr/local/lib/python2.7/dist-packages/gphoto2/backend.py", line 186, in _check_error
    raise errors.error_from_code(rval)
GPhoto2Error: Could not claim the USB device


As we turned on the camera, Ubuntu detected the camera and opened the camera device in the filemanager. To avoid automounting (temporarily disabling) I found this solution for my Ubuntu system:

$ gsettings set org.gnome.desktop.media-handling automount "false"

$ gsettings set org.gnome.desktop.media-handling automount-open "false"

We start again the spread configuration (after powering off both cameras and connecting the usb hub):

$ spread configure
...

Do you want to configure the target_page of your devices?
(Required for shooting with two devices) [y/N]: y
Setting target page on cameras
Please connect and turn on the device labeled 'odd'
Press any key when ready.

Ooops! Even after disabling automount with above commands, the camera was automounted! Then I just unmounted it in the file manager and continued "spread configure" afterwards by pressing any key. That worked:

Configured 'odd' device.
Please turn off the device.
Press any key when ready.

(Turning off the "odd"-camera)

Please connect and turn on the device labeled 'even'
Press any key when ready.

(Turning on the "even"-camera)

Configured 'even' device.
Please turn off the device.
Press any key when ready.

(Turning off the "even"-camera)

Do you want to setup the focus for your cameras? [y/N]: N
Configuration file written to '/home/ralf/.config/spreads/config.yaml'
Exception AttributeError: "'NoneType' object has no attribute 'gp_camera_exit'" in  ignored

Ooops! What does this mean?
After asking the developer: never mind, just continue...

Start your cameras.
Start Spreads web server:

$ spread --verbose web
spreadsplug.web: Starting scanning station server in "full" mode
huey.consumer.ConsumerThread: 1 worker threads
huey.consumer.ConsumerThread: Setting signal handler
huey.consumer.ConsumerThread: Huey consumer initialized with following commands
+ process_workflow

+ upload_workflow
+ transfer_to_stick
+ output_workflow
huey.consumer.ConsumerThread: Starting scheduler thread
huey.consumer.ConsumerThread: Starting worker threads
huey.consumer.ConsumerThread: Starting periodic task scheduler thread

spreadsplug.web.discovery: Starting discovery listener

Now you can access the Web GUI under port 5000:

http://localhost:5000

Create a workflow for your book and start scanning.

After scanning all pages, you end up with TIF-files in the directory
~/scans/{project_name}/done

Start scantailor to cut and process the images. You end up again with TIF-files in the directory
~/scans/{project_name}/done/out

Change into this directory and convert all TIF files to JPEG files:
$ convert *.tif -set filename: "%t" %[filename:].jpg

If "convert" is not installed, install imagemagick:
$ sudo apt-get install imagemagick

Finally you've got all pages as JPG-files.

Resize all JPGs to half the size:

$ mkdir ../jpg-middle
$ convert *.jpg -resize 50% -verbose -set filename: "%t" ../jpg-middle/%[filename:].jpg

Tuesday, 7 July 2015

Thanks to this post
I can provide examples for how to map a MAP with different types of keys and values with JPA annotations:

@ElementCollection
@CollectionTable(name = "TEST_MAP0", joinColumns = @JoinColumn(name = "DEPARTMENT"))
@Column(name = "value")
@MapKeyColumn(name = "key")
private Map<String, String> map0;

@ManyToMany(targetEntity = com.hibernate.elephants.Employee.class)
@JoinTable(name = "TEST_MAP1", joinColumns = @JoinColumn(name = "DEPARTMENT"), inverseJoinColumns = @JoinColumn(name = "value"))
@MapKeyColumn(name = "key")
private Map<String, Employee> map1;

@ElementCollection
@CollectionTable(name = "TEST_MAP2", joinColumns = @JoinColumn(name = "DEPARTMENT"))
@MapKeyClass(value = com.hibernate.elephants.Employee.class)
@MapKeyJoinColumn(name = "key")
@Column(name = "value")
private Map<Employee, String> map2;

@ManyToMany(targetEntity = com.hibernate.elephants.ParkingSpace.class)
@JoinTable(name = "TEST_MAP3", joinColumns = @JoinColumn(name = "DEPARTMENT"), inverseJoinColumns = @JoinColumn(name = "value"))
@MapKeyClass(value = com.hibernate.elephants.Employee.class)
@MapKeyJoinColumn(name="key")
private Map<Employee, com.hibernate.elephants.ParkingSpace> map3;