Managing your Android phone over SSH with Rsync
I was really excited when I got my hands on the Galaxy Nexus. After carrying around a symbian phone for years, an internet enabled, high resolution touchscreen phone was pretty much a geek’s wet dream to me. So I went home, activated it, plugged the USB cable in, and…. nothing. Googling around taught me that there was some crap going on with mtplib that I wasn’t really going to resolve. Folks on the Interwebs suggested installing an ftp-server app, and transfer things that way. I did this for a while, and it worked just fine. But the Linux-lover inside me knew there had to be a better way. Manually managing files across nodes is so 1970!
So what do you do when you want to sync a folder from your computer with some other computer? If you answered anything but “rsync”, go do your homework. If there’s two things you’ll find on any unix system, that’s an ssh daemon, and rsync. Not even Bash is as ubiquitous! But, of course, I clashed with what Google thought would be “fine” for Android. It might be Linux at heart, but the surface sure doesn’t look anything like Linux. Two really important things missing are daemons and a proper shell.
Thankfully, there are a few SSHd apps on the market, some better than others. There is also at least one app that allows you to rsync from your phone, which is cool, but I’d rather handle things on my computer while the phone sits, dunno, in my pocket, or somewhere else. But no matter what you use, you should really see that it has support for key based authentication.
I use SSHDroid myself, which is a great app. It will automatically launch the ssh daemon when a wifi connection is found, and shut down when it’s gone. It does key based auth too. The only downside is that it won’t automatically rescan your file system. QuickSSHD does this, but I found it to be very, very laggy. YMMV.
Ideally, you’d be able to install an SSH daemon as system service, instead of an app you have to manually start and activate. But I don’t see that happening on non-rooted devices. Send your letter bombs to Google.
Rsync
Unfortunately, none of the above apps ship with an rsync executable, and Android itself certainly could care less about such a thing, so you’ll have to get it yourself and install it on the phone, somewhere where your sshd can find and execute it.
This handy page describes the process to do just that.
Putting it all together
Once you have an ssh server running on your phone, it’s time to do great stuff with it. Below are some shell functions I wrote to help me manage things through a staging folder. You do things in this folder, like adding or removing files, and then push to your phone. When pulling files from the phone, a compressed and encrypted snapshot will be created of the staging tree (which will contain any new files you had on the phone).
You want to put these in your .zshrc or similar. I use ZSH myself, but this might just run fine in Bash. Let me know if it doesn’t.
| # {{{ Android phone sync (PS) and backup | |
| PS_BACKUP_DIR=~/Android/Backup # Where to keep backups | |
| PS_BACKUP_KEEP=60 # Days to keep backups | |
| PS_BACKUP_ENCRYPT_TO="0x00000000" # GPG key ID to sign and encrypt to | |
| PS_IP="192.168.0.100" # The IP your device will be found at | |
| PS_SSH="Nexus" # The SSH host of your phone, or an SSH alias | |
| PS_STAGING=~/Android/Sync # The staging folder for sync, sans trailing slash | |
| PS_SDCARD=/sdcard # The sync root on the phone, sans trailing slash | |
| PS_RSYNC_OPT="-vzuLr --no-perms" # Extra options to pass to rsync | |
| # Do not leave out -L or --no-perms | |
| # Android doesn't like symlinks or permission fiddling. | |
| function ps_probe() { | |
| ping -c 1 ${PS_IP} > /dev/null | |
| if [[ $? == 0 ]]; then | |
| $@ | |
| return 0 | |
| else | |
| echo "The phone could not be located on the network" | |
| return 1 | |
| fi | |
| } | |
| function ps_pull() { | |
| ps_probe && rsync ${PS_RSYNC_OPT} --exclude /Music/ ${PS_SSH}:${PS_SDCARD}/ ${PS_STAGING} && ps_backup_create | |
| } | |
| function ps_push() { | |
| ps_probe && rsync ${PS_RSYNC_OPT} --delete-after ${PS_STAGING}/ ${PS_SSH}:${PS_SDCARD} | |
| } | |
| function ps_backup_create() { | |
| t=$(mktemp -d) | |
| ts=$(mktemp -d) | |
| cp -a ${PS_STAGING}/* $t | |
| [[ $? != 0 ]] && echo "Failed to copy files to tmp folder." && return 1 | |
| fn="${ts}/$(date +%Y-%M-%d).tar.bz2" | |
| tar -cjpsf $fn ${t}/* | |
| [[ $? != 0 ]] && echo "Failed to create snapshot archive." && return 1 | |
| gpg2 -se -r $PS_BACKUP_ENCRYPT_TO $fn | |
| mv -f "${fn}.gpg" $PS_BACKUP_DIR/ | |
| rm -rf $t | |
| rm -rf $ts | |
| ps_backup_rotate | |
| } | |
| function ps_backup_rotate() { | |
| # TODO: Rotate | |
| find $PS_BACKUP_DIR -mtime $PS_BACKUP_KEEP -delete | |
| } | |
| function ps_backup_restore() { | |
| # TODO: Date parsing to restore specific backup. | |
| # Create a new backup first, if no current backup is found | |
| if [[ ! -f "${PS_BACKUP_DIR}/$(date +%Y-%M-%d).tar.bz2.gpg" ]]; then | |
| echo "Creating a backup first..." | |
| ps_backup_create | |
| fi | |
| if [[ -f "$1" ]]; then | |
| rm -rf ${PS_STAGING}/* | |
| gpg2 -d $1 | tar xjf - -C $PS_STAGING --strip-components 2 | |
| fi | |
| } | |
| # }}} |
Music
If you’re a console cowboy like me, chances are you use something like MPD to listen to your jams. I bet you’d like your hand crafted, carefully curated playlists on your portable jam-player, right? Well, I do, so I came up with the script below.
| #!/usr/bin/env zsh | |
| # Takes all my playlists from ~/.mpd/playlists, fixes them up, and creates a | |
| # folder for each, along with the music they reference. | |
| # The sync stage requires an sshd server to run on your phone, as well as the rsync executable. | |
| # - http://linux.wxs.ro/2011/08/05/rsync-your-android/ | |
| MPD_MUSIC_ROOT="${HOME}/Music" # Root of your MPD library | |
| MPD_PLAYLIST_ROOT="${HOME}/.mpd/playlists" # MPD playlist folder | |
| SYNC_MUSIC_ROOT="${HOME}/Android/Sync/Music" # Staging folder (receives all files prior to sync) | |
| PLAYLIST_ROOT="${SYNC_MUSIC_ROOT}/playlists" # Where to place playlist files (m3u) for sync | |
| PHONE_MUSIC_ROOT="/sdcard/Music" # Where the music will live on your phone | |
| PHONE_ADDRESS="Nexus" # Address to access your phone over ssh for sync | |
| # Output stuff to stdout if $DEBUG evaluates to anything | |
| # $ DEBUG=1 plsync.sh | |
| function debug() | |
| { | |
| [[ ! -z "$DEBUG" ]] && echo $* | |
| } | |
| # Sync stuff with phone over rsync+ssh | |
| function sync() { | |
| rsync -vzrL --delete-after --no-perms --no-t ${SYNC_MUSIC_ROOT}/ ${PHONE_ADDRESS}:${PHONE_MUSIC_ROOT}/ | |
| } | |
| # Ask for confirmation on something | |
| function confirm() | |
| { | |
| echo -n "$@ [y/N] " | |
| read answer | |
| for response in y Y yes YES Yes Sure sure SURE OK ok Ok kk; do | |
| [[ "_$answer" == "_$response" ]] && return 0 | |
| done | |
| return 1 | |
| } | |
| [[ ! -d "$PLAYLIST_ROOT" ]] && mkdir -p "$PLAYLIST_ROOT" | |
| for pl in ~/.mpd/playlists/*.m3u; do | |
| plname="$(basename ${pl})" | |
| plname="${plname%%.*}" | |
| plfolder="${SYNC_MUSIC_ROOT}/${plname}" | |
| plfile="${PLAYLIST_ROOT}/${plname}.m3u" | |
| [[ -f "$plfile" ]] && rm -f "$plfile" | |
| [[ -d "$plfolder" ]] && rm -rf "$plfolder" | |
| [[ ! -d "$plfolder" ]] && mkdir -p "$plfolder" && touch "$plfile" | |
| while read line; do | |
| f="${MPD_MUSIC_ROOT}/${line}" | |
| bn=$(basename "$f") | |
| if [[ -f "$f" && ! -f "${plfolder}/${bn}" ]]; then | |
| ln -s "$f" "$plfolder" | |
| echo "${PHONE_MUSIC_ROOT}/${plname}/${bn}" >> "$plfile" | |
| fi | |
| done < "$pl" | |
| done | |
| confirm "Would you like to sync with your phone? (Make sure sshd runs on the device)" && sync |