screaming.computer

When brainstorming for this project, I knew I'd eventually need to write a web scraper to gather text content for the screaming.computer. I did quite a bit of Googling, and none of the scraping options looked promising. The available PHP libraries tended to be wildly overcomplicated for my needs, or very out-of-date, or just unnecessarily difficult to use. There were also a bunch of cloud-based scrapers and a few browser plugins, but these have dependencies I'd rather avoid.

Discovering the PHP Simple HTML DOM Parser library was like striking gold. It's up-to-date, easy-to-use, and still quite fully-featured.

Install SimpleHtmlDom using composer

Having previously installed composer to manage another library, it's already available on my LAMP server.

Create a directory for the SimpleHtmlDom library:

mkdir /var/www/_simplehtmldom
cd /var/www/_simplehtmldom

Attempt to install the library stable version:

composer require simplehtmldom/simplehtmldom

Could not find a version of package simplehtmldom/simplehtmldom matching your minimum-stability (stable).

Attempt to install any available version of the library:

composer require simplehtmldom/simplehtmldom:*

Your requirements could not be resolved to an installable set of packages.
The requested package simplehtmldom/simplehtmldom * is satisfiable by simplehtmldom/simplehtmldom[2.0-RC2, dev-master] but these conflict with your requirements or minimum-stability.

Okay, fine, we'll install the Release Candidate by name:

composer require simplehtmldom/simplehtmldom:2.0-RC2

Clearly I don't know anything about how to use composer, but we have success nonetheless!

Scrape the web using PHP

This PHP fragment shows one basic use of SimpleHtmlDom — loading the CBC website and printing a list of all links on the page:

require_once ('/var/www/_simplehtmldom/vendor/autoload.php');
use simplehtmldom\HtmlWeb;

$webParser = new HtmlWeb();
$htmlDoc = $webParser->load ('https://www.cbc.ca/news/');
foreach ($htmlDoc->find ('a') as $anchor)
  echo $anchor->href, '<br>';

There are also some very helpful examples at the SimpleHtmlDom site.

The thermal receipt printer I am currently using doesn't natively supporting printing QR codes.

We can extend the escpos-php library to add image-based QR code support through the PhpQrCode library, as indicated in this tutorial by mike42 (the author of escpos-php).

These instructions assume a LAMP server with escpos-php has been configured per previous guides.

Install GD and PhpQrCode

Install the GD graphics library for PHP (verify your PHP version, alter GD library name as appropriate)

php --version
sudo apt-get install php7.2-gd
sudo service apache2 restart

Download the PhpQrCode library
Unzip it into /var/www/_phpqrcode

Subclass the escpos-php Printer class

We can create a new QrImgPrinter class which extends the escpos-php Printer class, but implements the qrCode function using image-based techniques rather than native printer support.

This code is based on the mike42 tutorial, but is updated to fix a few bugs and work with the current versions of PhpQrCode and escpos-php.

Create a new PHP file /var/www/_escpos/QrImgPrinter.php:

<?php
require_once ('/var/www/_escpos/vendor/autoload.php');
use Mike42\Escpos\Printer;
use Mike42\Escpos\EscposImage;

require_once ('/var/www/_phpqrcode/qrlib.php');


class QrImgPrinter extends Printer {

 function qrCode (string $content, int $ec = self::QR_ECLEVEL_L,
  int $size = 3, int $model = self::QR_MODEL_2)
 {
  // Validate inputs
  self::validateInteger ($ec, 0, 3, __FUNCTION__);
  self::validateInteger ($size, 1, 16, __FUNCTION__);
  self::validateInteger ($model, 1, 3, __FUNCTION__);
  if (strlen ($content) < 1)
    return;

  // Only Model 2 supported in phpqrcode
  $model = self::QR_MODEL_2;

  // Generate filename for temp file
  $tmpfname = tempnam (sys_get_temp_dir(), 'escpos-php');

  // Create QR code in temp file and print it
  QRcode::png ($content, $tmpfname, $ec, $size, 0, false);
  $img = EscposImage::load ($tmpfname, false);
  $this->bitImage ($img);

  // Delete temp file
  unlink ($tmpfname);
 }
}
?>

Use the QrImgPrinter class to print QR codes

In your existing receipt printing code, include the QrImgPrinter.php file and change your class instantiation from new Printer to new QrImgPrinter. Then call the qrCode member function normally to print a QR code as an image:

$connector = new FilePrintConnector ('php://stdout');
$printer = new QrImgPrinter ($connector);
$printer->initialize();
$printer->qrCode ('http://screaming.computer', Printer::QR_ECLEVEL_L, 6);
$printer->text ("\n"); // Flush
$printer->close();

Most thermal receipt printers are controlled using the ESC/POS protocol. This is a standard set of control codes to control font size and style, print images and barcodes, etc. (though not all printers support all features — some testing is generally required). The escpos-php library is a fantastic piece of software which wraps this protocol in an easy-to-use set of PHP classes.

Installing the escpos-php library is fairly easy, however it uses the composer dependency manager which I was previously unfamiliar with.

These are the steps I took to install escpos-php on my Linux server VM and print using my thermal receipt printer:

Install composer:

sudo apt install composer

Install the php-intl module:

sudo apt-get install php-intl
sudo service apache2 restart

Create a folder for escpos-php in your web directory, then install it:

mkdir /var/www/_escpos
cd /var/www/_escpos
composer require mike42/escpos-php

Create a PHP file printerTest.php in a suitable location:

require_once ('/var/www/_escpos/vendor/autoload.php');
use Mike42\Escpos\Printer;
use Mike42\Escpos\PrintConnectors\FilePrintConnector;

$connector = new FilePrintConnector ('php://stdout');
$printer = new Printer ($connector);

$printer->initialize();
$printer->text ("test\n");
$printer->close();

This PHP file will output the binary control codes to stdout. We run the program and pipe the output to our printer as follows:

php printerTest.php | lp -s -d THERMALPRINTER

In theory, we could use a CupsPrintConnector instead of the FilePrintConnector which would save us the step of redirecting the output. However, in my (very brief) test, the output seems to go into a print queue and must be manually released before it will print. This may have been a one-time hiccup or there may be a workaround, but using the FilePrintConnector avoided the issue and suits my needs for now.

Guide v1.6 / Updated 2020-02-08 / First published 2020-02-07 / echo (at) screaming (dot) computer

I experienced many problems trying to access my thermal printer from my Linux VM running in UnRAID. After many diagnostics and lots of troubleshooting, I identified a two-part solution:

1. Install a USB expansion card into the UnRAID server and pass it through for the exclusive use of the VM. This addresses low-level USB errors in the virtual machine.
2. Install a generic CUPS driver and use the lp command to send data to the printer. This addresses the issue where usblp won't mount the device.

POS58 thermal receipt printer

This particular thermal printer is listed as:

Manufacturer=GD32 Microelectronics
Product=POS58 USB Printer
USB device: Winbond Electronics Corp. Virtual Com Port
Vendor & Product ID: 0416:5011

It's sold under the name “58MM USB Thermal Receipt Printer, Symcode” on Amazon.ca.

USB diagnostics in Linux

Some miscellaneous Linux commands which are helpful for diagnosing USB-connected devices:

lsusb
lsusb -v
ls -l /dev/usb
usb-devices
lspci | grep USB
dmesg

Failure part 1: libusb errors in UnRAID VM

With the thermal printer plugged into the UnRAID system (prior to adding the USB expansion card) and passed through to the VM, the VM system log showed the following errors when booting:

libusb: error [op_set_interface] setintf failed error -1 errno 110
libusb: error [op_clear_halt] clear_halt failed error -1 errno 71

Under lsusb it appeared as a Winbond Electronics Corp. Virtual Com Port, and running usb-devices showed the device as a POS58 USB Printer.

The printer appeared to be mounted correctly by the usblp kernel module at /dev/usb/lp0.

Inital attempts to access the printer failed due to lack of user permissions. The lp0 device was part of the lp usergroup, so I added my current user to that group:

sudo usermod -a -G lp $USER
sudo reboot

Even with correct permissions, sending data to the printer always failed with I/O errors:

echo "test" >> /dev/usb/lp0

bash: echo: write error: Input/output error

...and the VM system log showed this error:

libusb: error [submit_bulk_transfer] submiturb failed error -1 errno=2

Solution part 1: Add a USB expansion card dedicated for VM use

Install your USB expansion card and boot your UnRAID server.

UnRAID [Tools] tab → [System Devices]

Find the PCI ID of your newly-installed card and save it for later use; it is in the form XXXX:XXXX, where the X's are hexadecimal digits. My expansion card appeared as:

IOMMU group 13: [1106:3483] 03:00.0 USB controller: VIA Technologies, Inc. VL805 USB 3.0 Host Controller (rev 01)

so the ID number is 1106:3483

WARNING: Do not get this ID number wrong. Be sure you have the ID of the correct device. Using the wrong ID in the following steps can cause serious issues for UnRAID. Editing the following settings can cause your UnRAID server to become unbootable. Proceed at your own discretion.

UnRAID [Main] tab → [Flash] device → [Syslinux Configuration] → click in the [unRAID OS] textbox

On the append line, add a vfio-pci.ids parameter with the ID of your USB expansion card; this causes UnRAID to avoid “claiming” this device on startup, allowing it to be passed through for the exclusive use of a VM.

After editing, my append line in the unRAID OS section looked as follows:

append initrd=/bzroot vfio-pci.ids=1106:3483

[Apply], [Reboot] your UnRAID server

UnRAID [VMs] tab → (your VM) → [Edit]

USB Controller: 3.0 (qemu XHCI)
Other PCI Devices: [check the box for your expansion card]
[Update]

If you had previous USB devices passed-through to the VM which are no longer connected, you may receive this error: “internal error: unknown pci source type 'vendor'” — The easiest way to resolve this is to reconnect the USB devices, deselect them in the VM configuration, then remove the devices.

UnRAID [VMs] tab → (your VM) → [Start]
UnRAID [VMs] tab → (your VM) → [Logs]

Watch the log window for errors. Mine booted without issue.

Using VNC to access the VM, open a terminal. Run lsusb to observe your USB expansion card.

Successfully resolved: boot-time libusb errors. USB ports are now functioning correctly.

Failure part 2: usblp fails to mount the thermal printer device

Plug the thermal printer into the USB expansion card, run ls /dev/usb to see the mounted device (expecting lp0 or similar), then test the device while watching the log:

echo "test" >> /dev/usb/lp0

If your printer prints the word “test,” congratulations, it works!

Unfortunately this was unsuccessful for me. Running ls /dev/usb came up empty. Curiously, usblp was not mounting the printer now even though it did so prior to using the expansion card.

Running dmesg revealed the following error:

usblp: can't set desired altsetting 0 on interface 0

My expansion card appeared to be successfully configured, but this particular model of USB printer seems to behave in quirky or non-standards-compliant ways.

Others have observed similar issues with this model of printer:

• ESC/POS Printer not detected by Raspberry Pi
• How to get USB thermal printer to work in Linux?

It appears that usblp is failing to initialize and mount the printer.

(If, in your case, you see the device is mounted but it still generates I/O errors, look into blacklisting usblp and try the CUPS solution, below.)

Solution part 2: Install CUPS driver, use lp command to print

* Solution found in this AskUbuntu post: Print via terminal without usblp.

In the Linux Mint GUI, open [Start Menu] → [System] → [Printers] → [+Add]

In the list of [Devices] there was an entry for “Unknown (Printer),” described as “A printer connected to a USB port.” This was the thermal printer.

Devices: Unknown (Printer)
[Forward]

Select printer from database: yes
Makes: Generic
[Forward]

Models: Raw Queue
[Forward]

Printer Name: THERMALPRINTER
[Apply]

Open terminal, test the printer:

echo "test" | lp -d THERMALPRINTER

The printer successfully printed “test” for me.

You can suppress the job ID messages on stdout using the -s option. See the lp man page for full lp command syntax.

Successfully resolved: Use a generic CUPS driver to access the USB printer, in situations where usblp fails to mount the device.