Jan 30, 2015

Installing Flexget as local user

I've been using Flexget for some quite time now and under Gentoo, and possibly other distros, there's a big flaw if you just install it using root. The main problem resides when you update Flexget, since it usually ends overwriting the python-exec and setup-tools binaries and breaking emerge, pip or any other important python binary.

The solution behind this problem is to install Flexget as a normal user, since when upgrading it shouldn't be able to touch any important binary.

Installing Flexget

As a prerequisite you need pip (at least the one for python2.7) and setup-tools installed as root. Once the prerequisites are met, you can install Flexget with the following:
pip2 install --user flexget

That will download, build and install all the needed components under the .local folder inside your user's home directory. The Flexget binary will be in $HOME/.local/bin

If you have Flexget already installed as root, you would need to uninstall it first, check that the prerequisites are met and then change to the user you want to install from and run the above command.

Updating Flexget

The update command when Flexget is installed as a normal user is the same as it would be as root but adding the --user flag:
pip2 install --upgrade --user flexget 

Sep 4, 2014

Configuring our UPS in Gentoo

If we have an UPS with a communication link (serial or usb), it can be configured in Linux with Network UPS Tools (or NUT).


You need to install the sys-power/nut package. If your UPS has an USB interface, make sure you install it with the usb flag.


Once installed, visit http://www.networkupstools.org/ and go to the Compatibility List. There you can filter your UPS Manufacturer to check the NUT driver you will be using. If your Manufacturer appears but your Device Model doesn't you can try to use the same driver as any other model like the one you have. In my case I have a PowerWalker VI 650 LCD that doesn't appear on the list, but since other models of PowerWalker are using the blazer_ser and blazer_usb drivers, I'm going to use the blazer_usb driver for this one.

Edit the configuration file /etc/nut/ups.conf to add the configuration of your UPS, mine is as follows:
 driver = blazer_usb
 port = auto
 desc = "PowerWalker VI 650"

Then you can try if NUT can find your UPS by running the upsdrvctl command:
# upsdrvctl start
Network UPS Tools - UPS driver controller 2.6.5
Network UPS Tools - Megatec/Q1 protocol USB driver 0.09 (2.6.5)
Supported UPS detected with mustek protocol
Vendor information unavailable
No values provided for battery high/low voltages in ups.conf

Using 'guestimation' (low: 10.400000, high: 13.000000)!
Battery runtime will not be calculated (runtimecal not set)
If the command runs without errors, you have your UPS successfully detected by NUT. Some drivers as the blazer_usb, will require extra commands to provide the battery charge and autonomy values.

Last step is configure an administration user name in /etc/nut/upsd.user. Once that is done, you can start the daemon and if you don't get any error, add it also to your runlevel.
# /etc/init.d/upsd start
 * Starting upsd ...
fopen /var/lib/nut/upsd.pid: No such file or directory
listening on port 3493
not listening on ::1 port 3493
Connected to UPS [PowerWalker]: blazer_usb-PowerWalker
# rc-update add upsd
 * service upsd added to runlevel default


Once we have the daemon started, we can monitor the UPS with the upsc command. This command requires the name we gave the UPS in the /etc/nut/ups.conf file and will give us all the information about our newly-installed UPS:
# upsc PowerWalker
battery.charge: 100
battery.voltage: 13.50
battery.voltage.high: 13.00
battery.voltage.low: 10.40
battery.voltage.nominal: 12.0
device.type: ups
driver.name: blazer_usb
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.version: 2.6.5
driver.version.internal: 0.09
input.current.nominal: 2.0
input.frequency: 50.1
input.frequency.nominal: 50
input.voltage: 231.5
input.voltage.fault: 231.5
input.voltage.nominal: 230
output.voltage: 231.5
ups.beeper.status: disabled
ups.delay.shutdown: 30
ups.delay.start: 180
ups.load: 9
ups.productid: 5161
ups.status: OL
ups.type: offline / line interactive
ups.vendorid: 0665

Jan 15, 2014

Run Flexget daemon at boot time (with init.d support)

Flexget version 1.2 and above add an option to run it as a daemon process instead of executing it several times via script or crontab. The good thing about this change is that now we can have an init script to start the daemon at boot time and forget about the needed crontab or script to run it.

Since this change isn't already on Gentoo's Portage main tree, I've opened Bug 498012 to submit the necessary changes to add it to Portage.

Applying the patch

You can find all the needed files as attachments in the submitted bug.
  • flexget-1.2.31: This file is the init.d script, it should be copied to /usr/portage/app-net/flexget/files, create the folder files if it doesn't exist.
  • flexget_conf: This file is the conf.d source needed for the init.d script to work, it should be copied to the same folder.
  • flexget-1.2.31.patch: This is the ebuild patch, needed to patch the actual flexget ebuild. To update the actual build, run the following:
    cd /usr/portage/app-net/flexget
    patch -p1 flexget-1.2.31.patch
  • The last step should be to regenerate the manifest file in /usr/portage/app-net/flexget, since we now have a modified ebuild, to do this, just execute the following inside that directory:
    cd /usr/portage/app-net/flexget
    ebuild flexget-1.2.31.ebuild manifest
Once you've finished applying the patch, you need to reinstall flexget to apply the new configuration. Then you just need to edit the contents of /etc/conf.d/flexgetd with your user and config file and you're ready to run flexget via init.d script.

Sample Configuration

# /etc/conf.d/flexgetd

# Options to pass to flexgetd at startup
# Default username to run flexgetd
# Flexget configuration file location
# Example: "/home/user/.flexget/config.yml"

Remeber that you can add the init.d script to boot time with rc-update.

Nov 26, 2013

Asus Transformer (TF101) bootloader fix

Playing with different ROMs or flashing funny stuff might cause your tablet to not boot correctly or always try to boot on recovery.

How to power it up?

You can try to power the tablet bypassing the bootloader with the following steps:
  1. Turn on the tablet using Vol Down + Power
  2. The tablet should display a couple of white text lines in the top, to let you know that you need to push Vol Up to enter recovery. Just wait there for it to timeout (its around 5 seconds)
  3. In the following screen, the tablet should show two icons (wipe data and android), make sure android is selected and either wait 10 seconds or hit the volume key to select it.
  4. The tablet should start normally again.
This is a workarround, so if you don't fix the problem, you will need to do all steps again every time you power up the tablet.

Whats the problem?

Most of the time, the issue is that the first string of the bootloader partition is corrupted or changed and the tablet either runs recovery or doesn't know what to run at all. To check if this is your table's issue, you need either adb or a terminal app in your device.

$ su
# cat /dev/block/mmcblk0p3
(more gibberish)

The solution

First of all, do this at your own risk. Second, check if the output of your boot partition has the 'boot-recovery' string, if it has any other string, this might not work at all. Now with the steps of the solution:

  1. Use either adb or a terminal/shell app with root.
    echo boot | dd of=/dev/block/mmcblk0p3 bs=1 seek=0
  2. Check again your boot partition, you should have something like the following
    cat /dev/block/mmcblk0p3
    (more gibberish)
  3. Reboot
The tablet should reboot now without problem as well as it should let you enter recovery without issues.

Nov 15, 2013

Restoring chat messages in LINE

As LINE and WhatApp user, I find the lack from LINE to be able to backup your chat history a big draw. Luckily for us, there's a workaround to fix this issue.

What do we need?

  • A rooted Android phone
  • Titanium Backup
  • SQLite Editor

Any other similar app should also do the trick. Before changing your phone, ROM or whatever, you need to create a backup of the LINE app, so we have the saved data of the app before doing anything.

How to restore

  1. Do a fresh install of LINE from the App Store, but don't open it right away.
  2. Restore LINE app data from the Titanium Backup backup.
  3. Open SQLite Editor and in the apps tab select LINE app.
  4. SQLite should open the app and show you the available .db files, select naver_line
  5. You should now see all the tables inside that .db file, select setting
  6. Once you're inside the table, seeing all the data it contains, select Menu > Delete All
  7. Open LINE app and login with your username
  8. You will receive a pop-up telling that the previous account will be deleted along with the chat history. Just click OK.
  9. Follow the rest of the setup process
  10. Once finished, you should be able to see all your saved chat history.
And that is, we now have our restored data in LINE.

Update: Another way to do it

If the above doesn't work for you, there's another way to do it with SQLite Manager
  1. As before, do a previous backup.
  2. Restore your backup on your new phone or ROM
  3. Use a root file explorer to go to /data/data/jp.naver.line.android/databases
  4. Copy naver_line to your PC and rename it as naver_line_old
  5. Do a fresh install and do the whole setup process
  6. Again use a root explorer to get the new version of naver_line and save it as naver_line_new on your PC
  7. In your PC, open naver_line_old and export chat_history table to a file.
  8. In your PC, open naver_line_new and import the exported file.
  9. Copy naver_line_new to /data/data/jp.naver.line.android/databases/naver_line
  10. Restart your phone.
With these steps, you should have all your chat history restored, but it won't be visible until the people in your chat conversations message you again.

Nov 6, 2013

Task management with Cron + Anacron

What is Anacron

Anacron is used to execute commands periodically, with a frequency specified in days. Its a good partner of any machine with cron jobs that it isn't running 24/7. For each job, Anacron checks if it has been executed in the last n days, where n is the period specified for that job. If the job hasn't been executed, Anacron will execute it.

Anacron configuration

Anacron configuration file is located in /etc/anacrontab and looks like this:
# format: period delay job-identifier command
## If you turn any of the following on, make sure that the
## regular cron daemon does not run them also. This could
## lead to double execution of the jobs.
1       5       cron.daily      run-parts /etc/cron.daily
7       10      cron.weekly     run-parts /etc/cron.weekly
30      15      cron.monthly    run-parts /etc/cron.monthly 

Tasks are represented in the following format:
period  delay   job-identifier  command

The job identifier should be unique for each job, as Anacron uses that name to create a file in /var/spool/anacron with the timestamp of the last time that task was executed.

Conflicts between Cron and Anacron

If the PC is restarted and Cron executes its daily/weekly/monthly routines, it is possible to execute them twice if also Anacron tries to execute them. To avoid this conflict, we can create a file inside the cron folders to update the timestamp of Anacron tasks and avoid running them again.

The simplest example should be to create a script called 0anacron, inside the cron.daily directory, that runs Anacron with the -u option, just to update the timestamps of Anacron's tasks:
/usr/sbin/anacron -u

If you have Anacron configured to execute different tasks in different time frames (such as one daily and one weekly), it is possible to append the task name or job-identifier to the -u option, so Anacron will only refresh that task. This way we can create one script in cron.daily to reflesh the daily Anacron task and one script in cron.weekly to refresh the weekly task.

Anacron on Suspend

Anacron will run automatically everytime you turn on (or reboot) your PC, but what happens when you just suspend it? Anacron won't run when you wake your PC from suspend, but there are a couple of things you can do to avoid this issue:

Use Cron

This might not be the most elegant solution but it works, just create an entry in your crontab that calls Anacron every X hours, where X should be a period short enough that would grant that your PC will be online in at least one period of a whole day. That way all daily tasks should be able to run without problems, for example:
# Run Anacron every two hours
* */2 * * * /usr/sbin/anacron -s

Use pm-utils

You need to configure pm-utils on your kernel, but it is possible to control several events, including wake from suspend. With a custom script on wake from suspend events, we could make a call to Anacron each time the PC wakes up, so we can ensure that all tasks run without problems. You should create a new executable script in /etc/pm/sleep.d. Any script found there on sleep or wake-up routines will be executed, so you can create a script like the following to execute Anacron each time the PC wakes up:
case $1 in
    /usr/sbin/anacron -s

Oct 22, 2013

Moving on to syslog-ng 3.4 (I)

syslog-ng is an open source implementation of the Syslog protocol for Unix and Unix-like systems. It extends the original syslogd model with content-based filtering, rich filtering capabilities, flexible configuration options and adds important features to syslog.

Upgrading to version 3.4 is as good as any other excuse to create a custom configuration and fit syslog-ng to our needs.

The basics

syslog-ng configuration file is pretty forward, the idea behind it is to set what you want to monitor, how you want to do it and where you want to save it. This is a basic configuration example:
@version: 3.4

# Global options
options {
# Create new dirs if needed
# log a MARK line every hour.
# log a STATS line every hour.
# Enable multi-thread

# Sources
source s_all { system(); internal(); };

# Destinations
destination d_syslog { file("/var/log/messages"); };

# Log
log { source(s_all); destination(d_syslog); };

Lets split the configuration in its respective parts to make it easier to understand.

Global Options

The global options configure syslog-ng basic features and behaviours. Its wise to read them all and add them to your configuration file if you want to configure something different from the default behaviour. You can check all available options in Chapter 9.2 of the Administration Guide.


Configure where does syslog-ng need to look, the basic configuration is the following:
source <name> { <source_type> };
Being <source_type> one of the following: internal(), file(), pacct(), pipe(), program(), sun-stream(), sun-streams(), syslog(), system(), tcp(), tcp6(), udp(), udp6(), unix-dgram(), unix-stream(). You can check what does each one do in Table 6.2 of the Administration Guide.


They aren't included in the example, but you can create filters to filter by facility, level, program or source amongst others. This will let you divide all the messages from the same source between the log files you want. For example, if you want to filter all ftp messages, you can create a filter like this:
filter f_ftp { facility(ftp); };
Then you could use that filter in a log statement to save those logs in the destination you choose.


Where you want to save your logs, either local or remote. In this case, we're gonna use a local file instead, but you can check all possible destinations (including SMTP) in Table 7.1 of the Administrator Guide.


Log statements assemble one or more of all the following statements (sources, filters and destinations) to build the chain that will be followed for every log we want. As said before, the basic is to select where to get the log (source), you can select how to parse it (filter) and where to store it (destination).

So, finishing our FTP monitor example, this should be the resulting code to create a log for FTP:
source s_all { system(); internal(); };
filter f_ftp { facility(ftp); };
destination d_ftp { file("/var/log/ftp.log"); };
log { source(s_all); filter(f_ftp); destination(d_ftp); };