Beebotron music player/recorder - openbox pipe menu

User avatar
johnraff
Sperminator
Posts: 199
Joined: Wed Oct 17, 2012 6:38 pm
Location: Japan
Contact:

Beebotron music player/recorder - openbox pipe menu

Unread post by johnraff » Sat Aug 31, 2013 6:56 am

This is the lastest version of a script I published on CrunchBang to access the excellent Beebotron site, get a list of all the bbc "listen again" music programmes available, filter out the stuff you don't want and make an openbox pipe menu to play your choice in radiotray (or other) or record it with ffmpeg. Comments in the file hopefully make it all clear, but feel free to ask here if something doesn't seem to make sense.

Don't forget to change the regular expression filters to match your taste in music, unless it's identical to mine...
Filters are applied to both entry names and descriptions, but only names are displayed.
Basic expression: '(string1|string2|etc)'

(watch out for long lines which might get folded here)

Code: Select all

#!/bin/bash
# beebotron.sh - a script to download a list of
# recent bbc radio programmes from beebotron.org
# and generate an openbox pipe menu
# version 12/04/11 added optional top menu items
# version 13/08/31 added recording option
# version 13/09/05 added some error checks
# requires curl, awk and ffmpeg (for recording)

# Usage: add this entry:
# <menu id="beebotron" label="BBC Music Programmes" execute="/path/to/beebotron.sh" />
# to your .config/openbox/menu.xml
# The first time it is run that day the cache file will be downloaded,
# which will make the menu slow to pop out. You can avoid that by adding:
# /path/to/beebotron.sh --refresh &
# to your openbox autostart(.sh) file.

# choose player for radio (radiotray will load new urls via dbus)
player=radiotray

# choose beebotron url to use
url=http://beebotron.org/public3/genremusic.html
# Other genres are: genrechildrens, genrecomedy, genredrama, genrefactual,
# genrelearning, genrenews, genrereligethics, genresport, genreweather
#  - substitute for genremusic above.
# If you live in the UK you can use the high quality links,
# which end with "hq" eg:
# url=http://beebotron.org/public3/genremusichq.html

cache_file=$HOME/.cache/beebotron_music # path to cache file
# (change name to match genre)

# RECORDING (needs ffmpeg to be installed)
recording='true' # 'true' to enable, any other value to disable recording
download_folder="$HOME/downloads/bbc" # store downloaded files here
# optional terminal to run ffmpeg in (see line 100)
terminal=urxvt # should support the -e option

#######################################

## FILTERS using regular expressions ##
# If you don't filter the list the menu will be very long.
# Entry descriptions are in the html file, but not displayed in the menu.

# Any entries (including descriptions) matching the expression
# will be put at the top of the menu and further filters will be ignored.
#top='(^My Favourite Prog Title|^This Too)'
top='(^World on 3|^World Routes|^Cerys on 6|^Don Letts|womad|WOMAD)'

# Any entries (including descriptions) matching the expression
# will be skipped.
#skip='([Bb]ach|[Bb]eethoven)'
#skip='a^' # nothing matches this
skip='(^Ace|^Beverley.*s World of Music|^Breakfast|^C2|^Chatback|^Chris Hawkins|^Composer of the Week|^Damien St|^Dress Circle|^Drivetime|^Graham Torrington|^Mornings |^Phil White|^Pipeline|^Rob da Bank|^Scott Mills |^Shaun Keaveny|^Simon Mayo |^Sue Davies|^The Early Show |^The Evening Show|^Through the Night|^Toddla T|^Tommy Sandhu)'

# Any entries (including descriptions) NOT matching the expression
# will be skipped.
keep='([Jj]azz|[Ww]orld|[Gg]lobal|[Aa]sia|[Aa]frica|[Rr]eggae|[Bb]hangra|[Bb]lues|r&b|R&B|[Rr]oots|[Ll]atin|India|Pakistan|Thai|Lao|Indonesia|Nathan Fake|Late Junction)'

##########################################################################
error_exit() {
    echo "$0 error: $1" >&2
    exit 1
}

[[ -t 0 ]] && {
    [[ $1 = '--debug' ]] && shift || {
        echo "This script is not intended to be called from a terminal.
Use the --debug option to overrule this behaviour.
Otherwise, see the comments in $0 for usage information"
        exit
    }
}

if [[ $recording = true ]]
then
    needed_apps='curl awk ffmpeg'
else
    needed_apps='curl awk'
fi
missing_apps=
for i in curl $needed_apps
do
    hash $i || missing_apps+=" $i"
done
[[ $missing_apps ]] && error_exit "Some necessary packages missing: $missing_apps"

[[ $1 = "--refresh" || ! -f $cache_file || $(( $(date +%s) - $(stat -c %Z "$cache_file") )) -gt 86400 ]] && {
curl -so "$cache_file" "$url" || error_exit "failed to download list from beebotron"
}
[[ $1 = "--refresh" ]] && exit

[[ $1 = "--record" ]] && {
    [[ $2 && $3 ]] || error_exit "not enough arguments for --record option"
    [[ $2 =~ ^http://(www|open\.live)\.bbc\.co\.uk(/[[:alnum:]_]+)+$ ]] || error_exit "url is not valid"
    prog_name="$3-$( date +%y%m%d )"
    mkdir -p "$download_folder" || error_exit "unable to make $download_folder"
    dl_file="$download_folder/$prog_name".wma
    stream_url=$( tr [[:upper:]] [[:lower:]] <<< $( curl -s $2 || error_exit "curl failed to fetch $2" ) | awk 'BEGIN{RS="<ref href=\"";FS="\"";}NR!=2{next}{sub(/^mms/,"mmsh",$1);print $1;}' )
    # remove '$terminal -e' from next line to run ffmpeg in background
    $terminal -e ffmpeg -i "$stream_url" -vn -acodec copy "$dl_file"
    exit
}

awk -v TOP="$top" -v SKIP="$skip" -v KEEP="$keep" -v PLAYER="$player" -v RECORDER="$0 --record" -v RECORDING="$recording" '
function submenu(name,url) {
    return "\
\n    <menu id=\"" name "\" label=\"" name "\">\
\n        <item label=\"PLAY\">\
\n            <action name=\"Execute\"><command>" PLAYER " \"" url "\"</command></action>\
\n        </item>\
\n        <item label=\"RECORD\">\
\n            <action name=\"Execute\"><command>" RECORDER " \"" url "\" \"" name "\"</command></action>\
\n        </item>\
\n    </menu>"
}
function item(name,url) {
    return "\
\n    <item label=\"" name "\">\
\n        <action name=\"Execute\"><command>" PLAYER " \"" url "\"</command></action>\
\n    </item>"
}

BEGIN {
    MENU=TOPMENU=""
    RS="<a href=\""
    FS="\">"
}
/DOCTYPE/ {next}
/http:\/\/beebotron.org/ {exit}
# $1 is the url, $2 the name+description
$2 ~ TOP {
    sub(/<\/a>.*$/,"",$2)
    if (RECORDING == "true") {
        TOPMENU = TOPMENU submenu($2, $1)
    }
    else {
        TOPMENU = TOPMENU item($2, $1)
    }
    next
}
$2 ~ SKIP {next}
$2 !~ KEEP {next}
{
    sub(/<\/a>.*$/,"",$2)
    if (RECORDING == "true") {
        MENU = MENU submenu($2, $1)
    }
    else {
        MENU = MENU item($2, $1)
    }
}
END {print ("<openbox_pipe_menu>" TOPMENU "\n    <separator/>" MENU "\n</openbox_pipe_menu>")}
' "$cache_file"

exit
EDIT 130905 Added some sanity checks, thanks to Dr xaos. :idea:
Attachments
beebotron.sh.tar.gz
(2.62 KiB) Downloaded 215 times
Last edited by johnraff on Wed Sep 04, 2013 5:58 pm, edited 3 times in total.
All code is one.

machinebacon
Baconator
Posts: 10253
Joined: Thu Sep 16, 2010 11:03 am
Location: Pfälzerwald
Contact:

Re: Beebotron music player/recorder

Unread post by machinebacon » Sat Aug 31, 2013 9:10 am

Thanks dear Mister Raffles :) Loving the 'pre-loading' option for autostart.
..gnutella..

User avatar
xaos52
The Good Doctor
Posts: 190
Joined: Thu Aug 15, 2013 11:59 am
Location: Eernegem, Belgium

Re: Beebotron music player/recorder - openbox pipe menu

Unread post by xaos52 » Sun Sep 01, 2013 5:52 pm

Excellent and useful script, John.

I could find only one thing wrong with it, and believe me, I searched hard.

When running the program with arguments --record, I think you should add some validations for arguments $2 and $3:

if $2 starts with - or -- it will be interpreted as an option for curl and curl will fail
$3 is used as (part of) a filename, and should match a regex like [a-zA-Z_O-9]* to prevent special characters in the file name.

I agree it is far sought, but don't let me remind you of Murphy's law...

Enjoy your scripting,

xaos
Connected. Take this REPL, brother, and may it serve you well.

User avatar
johnraff
Sperminator
Posts: 199
Joined: Wed Oct 17, 2012 6:38 pm
Location: Japan
Contact:

Re: Beebotron music player/recorder - openbox pipe menu

Unread post by johnraff » Mon Sep 02, 2013 5:30 am

@xaos thank you very much for checking it over.

I was about to implement your first suggestion ($2, avoid - or --) right away, but on rethink: $2 is coming from the script itself (from beebotron), so should be a valid url already, in theory. (The script is not meant to be called with --record by the user, it calls itself when the RECORD option is chosen from the menu.) Maybe $2 should be checked against a url regex, and exit with an error message if it doesn't match? In fact that same check could be done before sending the url to the player too...

I'm less sure what to do about $3 - it's the item title taken from the html '<a href=url>TITLE</a>' so could contain any characters permitted on a web page. Unix filenames can contain any character - maybe the most useful action would be to sanitize the name to remove a few troublesome ones? There's no reason a filename shouldn't contain things like #,%,&.~ etc is there?

But as you say, Murphy is always watching...
All code is one.

User avatar
xaos52
The Good Doctor
Posts: 190
Joined: Thu Aug 15, 2013 11:59 am
Location: Eernegem, Belgium

Re: Beebotron music player/recorder - openbox pipe menu

Unread post by xaos52 » Mon Sep 02, 2013 8:33 am

There are people - like me - that don't use any menus, but launch scripts directly from the command line.
We expect a script to give us some help when we launch it with invalid arguments, or with an -h or --help option.
So a "usage" function would be helpful.

For such people it would be necessary to validate $2 - check if it starts with a dash, if it is a valid url... ( and $3? though this one is doubtful indeed - if the user wants a filename with funny characters in it, then let him. The user is boss after all).

Come to think of it, you should also check if the curl command exits with status zero - or check if stream_url is set.

Ideally any script should be resilient no matter what arguments and options you throw at it.
Connected. Take this REPL, brother, and may it serve you well.

User avatar
johnraff
Sperminator
Posts: 199
Joined: Wed Oct 17, 2012 6:38 pm
Location: Japan
Contact:

Re: Beebotron music player/recorder - openbox pipe menu

Unread post by johnraff » Mon Sep 02, 2013 2:27 pm

If you launch this script from the command line, you'll get a bunch of xml which isn't all that useful except for making an openbox pipe menu.

otoh if you launch it with the --record option it will start up ffmpeg and record the url for you. Even in this case, though, the awk line that extracts stream_url from the asx file is tweaked to work with the bbc. Other sites use 'ENTRY' instead of 'Entry' for example. It wouldn't be that hard to hack together your own ffmpeg command for general-purpose recording. I don't think there's really much point in trying to use the script other than for making an openbox pipe menu to be honest.

All that said, I will incorporate most of your suggestions, especially the "usage" text. Many thanks for the feedback!
All code is one.

User avatar
xaos52
The Good Doctor
Posts: 190
Joined: Thu Aug 15, 2013 11:59 am
Location: Eernegem, Belgium

Re: Beebotron music player/recorder - openbox pipe menu

Unread post by xaos52 » Mon Sep 02, 2013 3:13 pm

I am running most scripts fom within emacs. So yes, I got me a load of xml files, from which I can copy/paste the url's I am interested in and launch tge record command.

Perhaps your script could be generalized to work for other WM's than openbox, and just spit out a list of URL's on stdout from which theuser can then grep for what he is interested in, while continuing to work as it is now when openbox is active.

BTW: anopenbox user will have problems with your script when curl or ffmpeg are not installed, and I don't think he will get a lot of feedback on what is going wrong.

I am not just critcising for the sake of critisizing, believe me.
Is is just that catching all possible errors well is an important, often neglected part of scripting.

Hiding the script from the user, like you are doing is an excellent strategy, but you must be sure that all possible error conditions that can arise are caught,and let the user know what went wrong, is what I tkink.

Enjoy your scripting...
Connected. Take this REPL, brother, and may it serve you well.

User avatar
johnraff
Sperminator
Posts: 199
Joined: Wed Oct 17, 2012 6:38 pm
Location: Japan
Contact:

Re: Beebotron music player/recorder - openbox pipe menu

Unread post by johnraff » Mon Sep 02, 2013 4:18 pm

Hey, all this input is really appreciated!

I was originally making this for #!, where curl comes as standard issue, but a check for curl and ffmpeg on the system will be a good idea.
xaos52 wrote:catching all possible errors well is an important, often neglected part of scripting
Agreed absolutely!

As to generating a list of urls for general use - I would humbly suggest going straight to the beebotron page with dillo or a cli browser. Of course you wouldn't get the filtering. I would be prepared to make an alternative version for another menu generator like 9menu, or even a terminal menu if there seemed to be a demand...
All code is one.

User avatar
xaos52
The Good Doctor
Posts: 190
Joined: Thu Aug 15, 2013 11:59 am
Location: Eernegem, Belgium

Re: Beebotron music player/recorder - openbox pipe menu

Unread post by xaos52 » Mon Sep 02, 2013 4:55 pm

As to generating a list of urls for general use - I would humbly suggest going straight to the beebotron page with dillo or a cli browser. Of course you wouldn't get the filtering. I would be prepared to make an alternative version for another menu generator like 9menu, or even a terminal menu if there seemed to be a demand...
Of course.

You don't have to do it for me. :razz:

Lets wait for others to chime in and see if there would be interest for that.
Connected. Take this REPL, brother, and may it serve you well.

User avatar
johnraff
Sperminator
Posts: 199
Joined: Wed Oct 17, 2012 6:38 pm
Location: Japan
Contact:

Re: Beebotron music player/recorder - openbox pipe menu

Unread post by johnraff » Wed Sep 04, 2013 6:03 pm

Script now updated with various error checking, and a little message for people who like to open openbox pipe menus in emacs...

Thanks Dr. for all your suggestions!

Of course this is basically just entertainment and no critical system work is being done, but it was fun putting all that extra stuff in anyway.
All code is one.

Post Reply