August 27, 2020

UPS for Ubuntu 20

Recently purchased two new UPS, one for Windows, another one for Ubuntu workstation. (UPS: https://en.wikipedia.org/wiki/Uninterruptible_power_supply)

Model: APC UPS, 1500VA UPS Battery Backup & Surge Protector, BX1500M

(https://www.amazon.com/gp/product/B06VY6FXMM/)

APC UPS, 1500VA UPS Battery Backup & Surge Protector, BX1500M Backup Battery, AVR, Dataline Protection and LCD Display, Back-UPS Pro Uninterruptible Power Supply

Plugged in the workstation, mid-size LCD monitor, USB hub with external power, having several devices with it, and 8 bay USB external drive.  UPS shows it can run on battery for about 60 min with everything turned on.

With recent frequent storms, power goes out briefly occasionally and this will save me from the trouble while I'm working on my computer.


S/W on Ubuntu

  • apcupsd: daemon process communicating with APC UPS
  • apcupsd-cgi: providing Web UI for APC UPS status
  • There was an old GUI application, but it no longer runs on Ubuntu 20, and not supported.

Install

$ sudo apt-get install apcupsd

If Web UI is desired, install apache:

$ sudo apt-get install apache2 apcupsd-cgi

$ apt list --installed | grep apcupsd

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

apcupsd-cgi/focal,now 3.14.14-3build1 amd64 [installed]
apcupsd-doc/focal,focal,now 3.14.14-3build1 all [installed,automatic]
apcupsd/focal,now 3.14.14-3build1 amd64 [installed]

 

I installed apcupsd-cgi although web server is not installed.  This is to analyze the CGI programs to see if I can build a front-end app without web server.  It turns out that the CGI programs are not necessary to build a GUI front-end.  I'll explain this in later section of this posting.

 

Files

Configuration files:
/etc/apcupsd/apcupsd.conf   # main
/etc/apcupsd/hosts.conf     # if other computers use this UPS
/etc/apcupsd/multimon.conf  # fields to use in CGI

Executables are in this directory,
/etc/apcupsd      
/sbin/apcaccess
/sbin/apctest


Logs,
/var/log/apcupsd.events

CGI Files,
If Apache is installed, the Web UI screen can be accessed with this URL:
http://localhost/cgi-bin/apcupsd/multimon.cgi


/usr/lib/cgi-bin/apcupsd/multimon.cgi    #
/usr/lib/cgi-bin/apcupsd/upsfstats.cgi   # works as /sbin/apcaccess
/usr/lib/cgi-bin/apcupsd/upsimage.cgi    # graph generator
/usr/lib/cgi-bin/apcupsd/upsstats.cgi    #

 

Set up

For my environment, there is only one computer is using this UPS, attached via USB, and email is not configured.

$ sudo vi /etc/apcupsd/apcupsd.conf
UPSNAME APC_BX1500M
UPSCABLE usb
UPSTYPE usb
BATTERYLEVEL 5  # shutdown at battery level percentage
MINUTES 5       # remaining runtime in minutes in case of power failure to shutdown
DEVICE          # blank entry for USB

$ sudo vi /etc/default/apcupsd
ISCONFIGURED=yes

$ sudo cp /etc/apcupsd/apccontrol.orig /etc/apcupsd/apccontrol 

$ sudo systemctl restart apcupsd.service


CGI output analysis

All the CGIs come with the apcupsd-cgi package can run in console with proper environment variable set up.  The output is stored at:

https://gist.github.com/keithkim/9210a84d68730f1a71c3cffe4fd4e681

(short URL, https://bit.ly/2QyVTvD)


Upon examining the output HTMLs from the CGIs, it's nothing but providing nice looking web interface by parsing output from running apcaccess, and the log.

By issuesing man apcaccess you will see what the apcaccess output fields mean.

To build a simple GUI app just like gapcmon (see below under Reference), parsing this output is sufficient.  To provide GUI configuration screen, then it needs to provide a form and updating the configuration file, restarting the daemon process -- this will require privileged access level.

Idea - gnome shell extension would be nice, or wxPython or Tk app is good also.


Reference



August 26, 2020

Voilà for Jupyter

 

https://blog.jupyter.org/and-voil%C3%A0-f6a2c08a4a93

"Voilà turns Jupyter notebooks into standalone web applications."

$ conda install voila -c conda-forge

 

August 20, 2020

Misc Linux Tips: anaconda, vim, screen

Anaconda

Channels
  • Show all channels, $ conda config --show channels
  • Adding a channel, $ conda config --add channels https://channel_url
    $ conda config --set show_channel_urls yes
  • Remove channel, $ conda config --remove channels https://channel_url
Anaconda Config commands:
https://docs.conda.io/projects/conda/en/latest/commands/config.html

 

Vim 

copy & Paste

https://linuxize.com/post/how-to-copy-cut-paste-in-vim/

Y=Yank

yy: copy current line

3yy: copy 3 lines (or any number)

y$: copy til to the end of the line

y^: copy from the start of the line

yw: copy until the end of the word, e.g. 12345[6]789 --> 6789 (6 is where cursor is)

yiw: copy the current word

dd: delete the line

3dd: delete 3 lines

d$: delete until the end of the line

p: paste

<visual mode>

v: visual mode

V: visual line mode

Ctrl+v: visual block mode

y: copy

d: delete

 

Screen

http://www.softpanorama.org/Utilities/Screen/screen_logging.shtml

screen -r   # reattach
screen -ls  # list sessions

- Logging: Ctrl-a, H
- screen output: Ctrl-a, h


August 18, 2020

Linux Antivirus

This post is about using free antivirus programs on Ubuntu/Linux.

What doesn't work:

  • Sophos - couldn't install.  Downloaded, unzipped, no installer for Linux.
  • Comodo - .deb file downloaded, requires libssl0.9.8.  You could force install that version, but I decided not to.
  • F-prot - EOL, no uninstaller, no instruction to remove.

 What works:

  • ClamAV
  • Check Rootkit
  • Rootkit Hunter

Unfortunately, no real-time detection is available with non-commercial s/w.


ClamAV 

$ sudo apt-get install clamav clamtk

Comes with GUI.  When scanning, GUI could look frozen.


Start, Stop
$ sudo systemctl start|stop clamav-freshclam

Disable,
$ sudo update-rc.d clamav-freshclam disable


Configure
$ sudo dpkg-reconfigure clamav-freshclam


Command line
$ sudo clamscan --infected --remove --recursive
$ clamscan -r-i --bell ~/Downloads

Signature DB path= /var/lib/clamav/


 

 

 

Check Root Kit

$ sudo apt-get install chkrootkit

$ sudo chkrootkit


Rootkit Hunter

$ sudo apt-get install rkhunter

$ sudo vi /etc/rkhunter.conf.local

and add "PKGMGR=DPKG" for Ubuntu

$ sudo rkhunter -c


August 17, 2020

Visual Studio Code PyLint issue

Problem: For Visual Studio Code, valid Python code has red-squiggles.

 

Visual Studio Code is a great tool for python programming.  Recently, I found a strange issue: a valid python code line has red squiggles.  It is a method call of some package I installed, and VSC correctly pops up available methods for the package fine.  It's there, why red squiggle then?

This red squiggle is work of Linting telling me that something is wrong.  VSC supports different Lint programs, and PyLint is one of them that works for me the best. 

But this red squiggle for valid method -- this turns out to be a security feature: Pylint does not import non-stdlib C extensions as it is a security risk.

 

Solution:

Tell Pylint to load the extension anyways.

1. File → Preferences → Settings → User settings tab

2. Type in "python.linting.pylintArgs"  This will bring up Pylint Argument setting.

3. Click on "Add Item" button and add this line:

--unsafe-load-any-extension=y

All working well now.


References

August 9, 2020

[Note] Ada on Ubuntu 20

Installing Ada


Apt has two packages.  GNAT-GPS (IDE) didn't work properly for me:
  • gnat-gps - integrated development environment for C and Ada
  • gnat - GNU Ada compiler

Get this file from here and install instead -- it will install everything in one directory:
https://www.adacore.com/community


Compiling in console

$ vi helloworld.adb

with Ada.Text_IO;
procedure Hello is
begin
    Ada.Text_IO.Put_Line("Hello, world!");
end Hello;

$ gnatmake helloworld.adb
$ ll
total 52
drwxrwxr-x 2 kkim kkim  4096 Aug  8 20:02 ./
drwxrwxr-x 4 kkim kkim  4096 Aug  8 20:00 ../
-rwxrwxr-x 1 kkim kkim 30000 Aug  8 20:01 helloworld*
-rw-rw-r-- 1 kkim kkim   211 Aug  8 20:02 helloworld.adb
-rw-rw-r-- 1 kkim kkim  1662 Aug  8 20:01 helloworld.ali
-rw-rw-r-- 1 kkim kkim  1640 Aug  8 20:01 helloworld.o
$ ./helloworld
Hello, world!


 
Tutoria, misc
 

Open Source Ada projects

August 8, 2020

[Note] Kinect2 on Ubuntu

This is NOT about XBox-360 Kinect.

Environment: Ubuntu 20.04

Kinect v2, XBox One version + USB adapter for PC

For Win10 and more info, see - https://blog.keithkim.com/2020/02/kinect2-on-windows-motion-detection.html


Install

The whole step is just about downloading, compiling and installing libfreenect2

  • Requires gcc version 8
  • Requires CUDA sample code
  • Requires a lot of apt packages

[1] multiple gcc versions

Default was version 9.  I had to install version 8 and made it as default.

$ sudo apt -y install gcc-7 g++-7 gcc-8 g++-8 gcc-9 g++-9
$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 7
$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 7
$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 8
$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-8 8
$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 9
$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-9 9

# select v8:

$ sudo update-alternatives --config gcc
$ sudo update-alternatives --config g++

# check version

$ gcc --version
$ g++ --version

[2] install CUDA sample code

See, https://blog.keithkim.com/2020/08/note-cuda-on-ubuntu-2004.html

[3] Install lots of APT packages

$ sudo apt-get install build-essential cmake autoconf pkg-config doxygen \
 libusb-1.0-0-dev libtool libudev-dev \
 libturbojpeg libjpeg-turbo8-dev \
 mesa-common-dev libgl1-mesa-glx \
 freeglut3-dev libxrandr-dev libxi-dev libva-dev libva-drm2 \
 openni2-utils

[4] compile libfreenect2

If want to disable CUDA, change cmake line as below,

$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/freenect2 -ENABLE_CUDA OFF

If CUDA is installed, and you have the sample, type this before make to point where the sample is installed,

$ export CPATH=$CPATH:$HOME/NVIDIA_CUDA-10.1_Samples/common


$ git clone https://github.com/OpenKinect/libfreenect2.git
$ cd libfreenect2
$ mkdir build && cd build
$ cmake ..
$ make
$ make install
$ sudo cp ../platform/linux/udev/90-kinect2.rules /etc/udev/rules.d/
 

# Unplug, plug in Kinect first
# you may have to add yourself to 'plugdev' group.

$ ./bin/Protonect

 

Closing

 Using it in Python, OpenCV will be another posting.


Reference

 

August 7, 2020

[Note] CUDA on Ubuntu 20.04

Install Options

A couple of ways to install CUDA:

  1. get everything from NVIDIA CUDA site
  2. apt-get
  3. docker

 

For the latest CUDA version,  do method #1.  For easier way, #2.  I chose to do #2.  But after went through it, I'm not sure if it's really easier. 

And #2 method, the CUDA version is 10.1.  I haven't tried docker method.

 

Install

Checking driver version:

$ nvidia-smi


Fri Aug  7 18:35:38 2020      
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.100      Driver Version: 440.100      CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0   Quadro P2000        Off  | 00000000:09:00.0  On |                  N/A |
| 53%   47C    P8     7W /  75W |    663MiB /  5057MiB |     17%      Default |
+-------------------------------+----------------------+----------------------+
...


Install packages using apt-get

$ sudo apt-get install build-essentials nvidia-cuda-toolkit

Check nvcc version:

$ nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243


Install cudnn

  1. Download cudnn, https://developer.nvidia.com/cudnn
  2. uncompress
  3. sudo cp cuda/include/cudnn.h /usr/lib/cuda/include/
    sudo cp cuda/lib64/libcudnn* /usr/lib/cuda/lib64/
    sudo chmod a+r /usr/lib/cuda/include/cudnn.h /usr/lib/cuda/lib64/libcudnn*


Install sample codes

Using apt-get, it doesn't come with sample codes.  For installing Kinect2 (will be on another posting), the sample is needed.  Install only if you need it.

Download from github, https://github.com/NVIDIA/cuda-samples/tree/10.1.2 

And store it anywhere you want.  Note that some other dependent code may look for it in a certain path, such as /usr/local/cuda-X-Y...


A couple more things to do:

[1] 

nvcc is installed in /usr/bin/  -- examples and some programs I use (sorry didn't write it down and forgot what it was) looks for it in /usr/local/cuda/bin.  Create a sym link there.

[2] vi .vashrc

export LD_LIBRARY_PATH=/usr/lib/cuda/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/lib/cuda/include:$LD_LIBRARY_PATH


Reference

 

CUDA Samples

  1. https://docs.nvidia.com/cuda/cuda-samples/index.html
  2. https://github.com/NVIDIA/cuda-samples/tree/10.1.2
  3. https://www.pugetsystems.com/labs/hpc/How-To-Install-CUDA-10-together-with-9-2-on-Ubuntu-18-04-with-support-for-NVIDIA-20XX-Turing-GPUs-1236/#step-6-test-cuda-by-building-the-samples-from-source-for-both-cuda-92-and-cuda-100


Nvidia Docker

 

August 2, 2020

SikuliX on Linux - vision based automation

http://sikulix.com

It's a vision based automation tool, you give an image, it will find a match and do automation.  It's a great open source tool.

I used this on Windows, and now my main OS is Ubuntu, and this is some information and note on it.  It uses OpenCV and Tesseract OCR.

https://avleonov.com/wp-content/uploads/2017/02/script_window.png

Install On Ubuntu 20.04

Installing and using SikuliX on Window is simple and straightforward.  On Linux (Ubuntu 20.04 for my case), it needs little bit more work.

Requirements
  • OpenCV
  • Tesseract OCR
  • Java 11 or later

1. Download SikuliX

Download it from https://raiman.github.io/SikuliX1/downloads.html
or do
$ wget https://launchpad.net/sikuli/sikulix/2.0.4/+download/sikulixide-2.0.4.jar

2. Install Java 11 or later
  • This will not work with Java8. 
  • OpenJDK works fine.
  • Dealing with multiple Java versions: I have Java 8, 11, 14, and Oracle JDK8 as well.  My default Java is set to 8.  To run this software with Java 14, I used bash script, and added using Alacarte (https://en.wikipedia.org/wiki/Alacarte)
3. Install/Set up OpenCV, Tesseract OCR

$ sudo apt-get install libopencv4.2-java libopencv4.2-jni
$ sudo apt-get install tesseract-ocr

$ cd /usr/lib/jni/
$ sudo ln -s libopencv_java420.so libopencv_java.so


4. Bash script

#!/usr/bin/bash
JAVA_HOME=/usr/lib/jvm/java-14-openjdk-amd64
SIKULIX_HOME=/opt/sikulix
SIKULIX_JAR=sikulixide-2.0.4.jar
LOG=$SIKULIX_HOME/sikulix.log
$JAVA_HOME/bin/java -jar $SIKULIX_HOME/$SIKULIX_JAR >> $LOG 2>&1


SikuliX Resources

Other Resources

Other opensource or free automation tools

July 29, 2020

[dev note] anaconda and uwsgi

conda search doesn't show uwsgi, and pip install fails with various python versions.

Finally found the solution after a couple of days of frustration --
https://anaconda.org/conda-forge/uwsgi

conda install -c conda-forge uwsgi

July 27, 2020

Joplin - Onenote, Evernote replacement

About 10 years or so ago, I wrote a posting on Evernote when it was new and shiny.  Back then it was revolutionary, before Onenote added feature for sharing across multiple devices.

After it started to charge more than I'm willing to pay, and limiting storage spaces, I stopped using it.  And OneNote started to offer the same features and I had/still have license for it.  OneNote is free now, I believe.  I stopped using it as well -- I didn't like some of its features and how it works.

Since then I've been using just text editor and share the note via cloud services, gave up finding alternative solutions.

There is Joplin - https://joplinapp.org/
Open Source, free, available for all OS and mobile devices, and sync via cloud services.  I like that I have the control how/where my notes are stored/sync'd.

Joplin Adds Experimental WYSIWYG Editor



Some notes on Ubuntu


If Joplin is installed through SNAP, due to SNAP's security model, it has some limitation where notes are stored/backed up for directory sync.

Notes are stored in /home/$USER/snap/joplin-james-carroll/13/.config/joplin-desktop
(See this by going to Tools > General)  I don't think this location can be changed -- at least I haven't found how.

If sync to directory, due to SNAP model, this directory cannot be outside of user home directory.  Sym link won't work.



Installer versions

If preferred, use Desktop installer version.  There is no restriction which directory you want to sync (backup).

wget -O - https://raw.githubusercontent.com/laurent22/joplin/master/Joplin_install_and_update.sh | bash

Installer version stores the notes in, ~/.config/joplin-desktop

For terminal version of Joplin, https://joplinapp.org/terminal/

$ NPM_CONFIG_PREFIX=~/.joplin-bin npm install -g joplin
$ sudo ln -s ~/.joplin-bin/bin/joplin /usr/bin/joplin


Terminal version stores the notes in, ~/.config/joplin

Removing Installer version

Desktop version
$ rm ~/.joplin
$ rm ~/.local/share/applications/appimagekit-joplin.desktop

Remove note files and configuration
$ rm ~/.config/joplin-desktop

Terminal version,
$ sudo rm /usr/bin/joplin
$ rm -rf ~/.joplin-bin

To remove note files,
$ rm ~/.config/joplin


Warning on Desktop Joplin on Ubuntu 20.04

This is for both SNAP and installer versions for Desktop Joplin.

Tools > Options > Application, there is "Show tray icon".  This doesn't work for me.  If checked, and close the Joplin window by click on "X" button, it disappears but stay in memory.  Only way to get it back is run Joplin again.  You have to File > Quit to properly quit the application.  If "Show tray icon" is not checked off, closing Joplin window by clicking on "X" button will quit the application.


[Note] Gitea in Docker


Gitea (https://gitea.io/en-us/) is like a light weight Gitlab.

I wrote on Gitea previously, running on my personal server,
https://blog.keithkim.com/2019/03/ownserver-gitea-host-own-github-gitlab.html


Again, my server goes up and down depends on the schedule I set it to.  It usually in sleep mode and wakes up according the the schedule I set.  My main computer is on almost all day but the server is not as it's not heavily used.

This posting is a note on installing Gitea in Docker.

Docker image page, https://hub.docker.com/r/gitea/gitea

Pull the image,
$ docker pull gitea/gitea

Gitea official shows using docker-compose to customize further, https://docs.gitea.io/en-us/install-with-docker/ -- for my use, default works for me.  No volume mapping, no DB.

Run it,

$ docker run --name gitea -p 3000:3000 -p 222:22 -d gitea/gitea
$ docker logs -f gitea


Access it,
http://localhost:3000

  1. Click on "Sign-in" at upper-right corner, and it'll show Initial Configuration page.
  2. Configure as needed (for my case/this posting, no mysql, external volume is mapped).
  3. And then register user -- that will be the admin.