demu.redhttps://demu.red/2021-06-28T21:00:00-04:00A place of geekeryAuto remove a spammer's bitbot quotes of yourself2021-06-28T21:00:00-04:002021-06-28T21:00:00-04:00demuretag:demu.red,2021-06-28:/blog/2021/06/auto-remove-a-spammers-bitbot-quotes-of-yourself/<h3 id="another day, another weechat /trigger">Another day, another weechat /trigger</h3>
<p>As is often the case on the internet, some people are the reason we can't have nice things.
Today's instance of ass hattery involves an instance of <a href="https://github.com/jesopo/bitbot">bitbot</a>, its <a href="https://github.com/jesopo/bitbot/blob/master/modules/quotes.py">quote</a> module, and a trolly user who continually fills the quote db with garbage.</p>
<h4 id="tl;dr">TL;DR …</h4><h3 id="another day, another weechat /trigger">Another day, another weechat /trigger</h3>
<p>As is often the case on the internet, some people are the reason we can't have nice things.
Today's instance of ass hattery involves an instance of <a href="https://github.com/jesopo/bitbot">bitbot</a>, its <a href="https://github.com/jesopo/bitbot/blob/master/modules/quotes.py">quote</a> module, and a trolly user who continually fills the quote db with garbage.</p>
<h4 id="tl;dr">TL;DR</h4>
<p>Bad user overuses bot quote commands <code>,grab</code> and <code>,qadd nick=<nick> not a funny msg</code> -- both spamming channels with the commands and filling the db with crap.</p>
<h3 id="triggers to the rescue_1">Triggers to the rescue</h3>
<div class="highlight"><pre><span></span><code>/trigger addreplace ATriggerNameHere print <span class="s2">""</span> <span class="s2">"</span><span class="si">${</span><span class="nv">server</span><span class="si">}</span><span class="s2">.</span><span class="si">${</span><span class="nv">tg_tag_nick</span><span class="si">}</span><span class="s2"> == YourShortNetworkName.BadUserNick && </span><span class="si">${</span><span class="nv">tg_message_nocolor</span><span class="si">}</span><span class="s2"> =~ ^,(qadd|grab) </span><span class="si">${</span><span class="nv">nick</span><span class="si">}</span><span class="s2">"</span> <span class="s2">""</span> <span class="s2">"/msg -server </span><span class="si">${</span><span class="nv">server</span><span class="si">}</span><span class="s2"> BotNick ,qdel </span><span class="si">${</span><span class="nv">nick</span><span class="si">}</span><span class="s2">"</span> <span class="s2">"ok"</span>
/trigger
<span class="c1">## Pick a trigger name</span>
addreplace ATriggerNameHere
print
<span class="s2">""</span>
<span class="c1">## fill in the network and user nick after the double equals.</span>
<span class="c1">## ${nick} could be hard coded for your nick, but this way it works for your current nick.</span>
<span class="s2">"</span><span class="si">${</span><span class="nv">server</span><span class="si">}</span><span class="s2">.</span><span class="si">${</span><span class="nv">tg_tag_nick</span><span class="si">}</span><span class="s2"> == YourShortNetworkName.BadUserNick && </span><span class="si">${</span><span class="nv">tg_message_nocolor</span><span class="si">}</span><span class="s2"> =~ ^,(qadd|grab) </span><span class="si">${</span><span class="nv">nick</span><span class="si">}</span><span class="s2">"</span>
<span class="s2">""</span>
<span class="c1">## Fill in the bitbot instances nick, see above re nick var.</span>
<span class="s2">"/msg -server </span><span class="si">${</span><span class="nv">server</span><span class="si">}</span><span class="s2"> BotNick ,qdel </span><span class="si">${</span><span class="nv">nick</span><span class="si">}</span><span class="s2">"</span>
<span class="s2">"ok"</span>
</code></pre></div>
<h4 id="for other tildeverse users">For other tildeverse users</h4>
<p>Here is the version you are probably here for ^_-</p>
<div class="highlight"><pre><span></span><code>/trigger addreplace tilde_no_jan6_quote print <span class="s2">""</span> <span class="s2">"</span><span class="si">${</span><span class="nv">server</span><span class="si">}</span><span class="s2">.</span><span class="si">${</span><span class="nv">tg_tag_nick</span><span class="si">}</span><span class="s2"> == tilde.jan6 && </span><span class="si">${</span><span class="nv">tg_message_nocolor</span><span class="si">}</span><span class="s2"> =~ ^,(qadd|grab) </span><span class="si">${</span><span class="nv">nick</span><span class="si">}</span><span class="s2">"</span> <span class="s2">""</span> <span class="s2">"/msg -server </span><span class="si">${</span><span class="nv">server</span><span class="si">}</span><span class="s2"> tildebot ,qdel </span><span class="si">${</span><span class="nv">nick</span><span class="si">}</span><span class="s2">"</span> <span class="s2">"ok"</span>
</code></pre></div>
<h3 id="explanation_1">Explanation</h3>
<p>This trigger first matches that the network and nick of a message are for the assigned net, and nick.
Then the message is tested for either quote bot triggers, and your nick.
If all are matched, the trigger will /msg the bot to delete the last quote attributed to your nick.
It also happens to work 'silently' in that the offender may someday pull out their hair wondering why they quotes aren't there XD.</p>
<h3 id="links">Links</h3>
<p><a href="https://github.com/jesopo/bitbot">bitbot</a><br/>
<a href="https://github.com/jesopo/bitbot/blob/master/modules/quotes.py">quote</a> </p>A Notational Velocity Stopgap Solution for Linux2021-04-12T12:00:00-04:002021-04-12T12:00:00-04:00demuretag:demu.red,2021-04-12:/blog/2021/04/a-notational-velocity-stopgap-solution-for-linux/<h3 id="notational velocity's simplicity">Notational Velocity's Simplicity</h3>
<p><a href="https://notational.net/">Notational Velocity</a> was a great tool I started using back in the days of yore, when I was on iBook running Mac OS X 10.4.
Its simplicity, latter iterated on by <a href="https://brettterpstra.com/projects/nvalt/">nvALT</a>, was a superb.
At its core you have three parts: A search box, a …</p><h3 id="notational velocity's simplicity">Notational Velocity's Simplicity</h3>
<p><a href="https://notational.net/">Notational Velocity</a> was a great tool I started using back in the days of yore, when I was on iBook running Mac OS X 10.4.
Its simplicity, latter iterated on by <a href="https://brettterpstra.com/projects/nvalt/">nvALT</a>, was a superb.
At its core you have three parts: A search box, a notes list, and a view of the selected note.
By typing in the search box you limit the listed notes to ones whose title/contents matched.
When you selected a note you started editing it, and if there was no match you started editing a new note.
Everything after that feature set is extras.</p>
<p>Due to how simple this setup is, it is also trivial to use :w
your file syncing method of choice <sup>(<a href="https://syncthing.net/">syncthing</a> for me)</sup> to have your notes in other places.</p>
<h3 id="linux">Linux</h3>
<p>There have been a number of kludgey, mostly GUI alternatives to Notational Velocity.
Sadly many of the alternatives I tried attempt to do extras in an annoying way (forcing .md), didn't work well (glitchy), etc.</p>
<p><a href="https://pypi.org/project/nvpy/">NVpy</a> was one of the contenders, and at the time -- a few years ago now -- it just didn't function well.</p>
<p><a href="https://www.qownnotes.org/">QOwnNotes</a> is one I came across recently and seems like it would be decent if you were just starting out moving to this note system. That being said, it does kind of do a <em>lot</em> at once.
Also, it does have some hiccups like requiring a restart after changing settings...</p>
<h4 id="previous solution">Previous Solution</h4>
<p>Until recently I had been using <code>tv3</code> -- a resuscitation of <a href="https://github.com/vhp/terminal_velocity">Terminal Velocity</a> -- to fill my note taking need.
<code>tv3</code> was a fork of the project to update for python3, but the original tv3 fork recently was deleted from github.
Prior to <code>tv3</code>, there had already been one repostitory trade off for a new maintainer of Terminal Velocity.</p>
<p>Anyway, Terminal Velocity had a nice little ncurses UI that gave you the search box and list, and would drop you into your <code>$EDITOR</code> when you selected your note.</p>
<h3 id="finding a new option_1">Finding a New Option</h3>
<p>In my most recent look into alternatives, I did find what will surely be even better than my final solution later on -- <a href="https://github.com/alok/notational-fzf-vim">notational-fzf-vim</a>.
Unfortunately notational-fzf-vim has more dependencies than I want my vimrc to require...
Currently my vimrc can be dropped on a new workstation and automatically <a href="https://github.com/VundleVim/Vundle.vim">vundle</a> install my plugins, and this would break that.
Down the road I might consider writing a per machine test for my vimrc, but not for now.</p>
<h3 id="a new stopgap solution">A New Stopgap Solution</h3>
<p>While what I came up with is lacking a few things, such as searching the contents of a file at the same time it is doing file name searches, it seems pretty decent for a few lines of code.
Here is the excerpt containing my shell function:</p>
<div class="highlight"><pre><span></span><code><span class="c1">### Note ### {{{</span>
<span class="c1">## An aproximation of Notational Velocity and Terminal Velocity</span>
<span class="c1">## gensub currently replacing</span>
<span class="k">function</span> note<span class="o">()</span> <span class="o">{</span>
<span class="nb">pushd</span> ~/Notes <span class="m">1</span>>/dev/null <span class="o">&&</span> <span class="se">\</span>
vim <span class="k">$(</span>fzf -i --cycle --reverse --preview-window<span class="o">=</span>down --preview<span class="o">=</span><span class="s1">'cat {}'</span> --print-query <span class="p">|</span> gawk <span class="s1">'END{if($0 !~ /.txt$/){$0=gensub(" ","_","g",$0) ".txt"}; print $0}'</span><span class="k">)</span>
<span class="nb">popd</span> <span class="m">1</span>>/dev/null
<span class="o">}</span>
<span class="c1">### End Note ### }}}</span>
</code></pre></div>
<h4 id="function explained">Function Explained</h4>
<p>Starting from the edges and working in:</p>
<ul>
<li><code>pushd</code>/<code>popd</code> - ensures we iterate over the notes directory or exit</li>
<li><code>vim</code> - which opens/makes the file returned</li>
<li><code>fzf</code> - to give us a nice Notational Velocity like UI and limit search</li>
<li><code>awk</code> - using magic to return either the file, or the new file name with spaces replaced.</li>
</ul>
<h3 id="conclusion_1">Conclusion</h3>
<p>While there are some things missing, this certainly seems like a minimum viable replacement for my needs.
One which should have a low chance of no longer working in a few years too.</p>
<h3 id="links">Links</h3>
<p><a href="https://notational.net/">Notational Velocity</a><br/>
<a href="https://brettterpstra.com/projects/nvalt/">nvALT</a><br/>
<a href="https://syncthing.net/">syncthing</a><br/>
<a href="https://pypi.org/project/nvpy/">NVpy</a><br/>
<a href="https://www.qownnotes.org/">QOwnNotes</a><br/>
<a href="https://github.com/vhp/terminal_velocity">Terminal Velocity</a><br/>
<a href="https://github.com/alok/notational-fzf-vim">notational-fzf-vim</a><br/>
<a href="https://github.com/VundleVim/Vundle.vim">vundle</a><br/>
<a href="https://github.com/junegunn/fzf">fzf</a> </p>Making Weechat's Buflist Look Nicer2020-11-08T17:00:00-05:002020-11-08T17:00:00-05:00demuretag:demu.red,2020-11-08:/blog/2020/11/making-weechats-buflist-look-nicer/<p><strong>NOTE: Added 2021-02-24 update section</strong></p>
<h3 id="buflist">Buflist</h3>
<p>Not too <a href="https://weechat.org/files/releasenotes/ReleaseNotes-1.8.html">long ago</a>... ok, I guess about three and a half years ago now... <a href="https://weechat.org/">WeeChat</a> got one of its greatest features incorporated into the main code -- the buflist.
Prior to this, it was in the form of a script.</p>
<p>With this change, buflist …</p><p><strong>NOTE: Added 2021-02-24 update section</strong></p>
<h3 id="buflist">Buflist</h3>
<p>Not too <a href="https://weechat.org/files/releasenotes/ReleaseNotes-1.8.html">long ago</a>... ok, I guess about three and a half years ago now... <a href="https://weechat.org/">WeeChat</a> got one of its greatest features incorporated into the main code -- the buflist.
Prior to this, it was in the form of a script.</p>
<p>With this change, buflist became more powerful and versatile as well.</p>
<h4 id="fancyness">Fancyness</h4>
<p>For a while now I've had a fancy bit of logic in my buflist to show when a network is disconnected.
When the topic of having away and part status in the buflist came up while chatting, I did some refactoring to add those as well:</p>
<div class="highlight"><pre><span></span><code><span class="x">/set buflist.format.buffer "${format_number}${indent}${format_nick_prefix}${color_hotlist}${if:(${buffer.full_name} =~ ^irc)?${if:(${irc_server.away_time} > 0)?${color:yellow}>}${if:(${irc_channel.part} == 1)?${color:magenta}_}${if:(${irc_server.is_connected} == 0)?${color:*magenta}*}}${if:${type}==private?↪}${name}"</span>
<span class="x">## A Breakdown</span>
<span class="x">/set buflist.format.buffer </span>
<span class="x">"</span>
<span class="x"> ${format_number} ## buflist.format.number</span>
<span class="x"> ${indent} ## buflist.format.indent</span>
<span class="x"> ${format_nick_prefix} ## buflist.format.nick_prefix</span>
<span class="x"> ${color_hotlist}</span>
<span class="x"> ${if:(${buffer.full_name} =~ ^irc)? ## Checks that buffer is from irc plugin</span>
<span class="x"> ${if:(${irc_server.away_time} > 0)? ## Checks if an away time is set</span>
<span class="x"> ${color:yellow}> ## Adds color and indicator</span>
<span class="x"> }</span>
<span class="x"> ${if:(${irc_channel.part} == 1)? ## Checks part status</span>
<span class="x"> ${color:magenta}_ ## Adds color and indicator</span>
<span class="x"> }</span>
<span class="x"> ${if:(${irc_server.is_connected} == 0)? ## Checks connect status</span>
<span class="x"> ${color:*magenta}* ## Adds color and indicator</span>
<span class="x"> }</span>
<span class="x"> }</span>
<span class="x"> ${if:${type}==private? ## Checks if privmsg</span>
<span class="x"> ↪ ## Adds indicator</span>
<span class="x"> }</span>
<span class="x"> ${name} ## buflist.format.name</span>
<span class="x">"</span>
</code></pre></div>
<h5 id="explanation">Explanation</h5>
<p>With this setting, an <code>/away</code> irc buffer gets colored yellow and marked with a <code>></code>; a <code>/part</code>-ed chan gets colored magenta and marked with a <code>_</code>; and a disconnected irc buffer gets bold magenta and marked with a <code>*</code>.
The order of the <code>if</code> test also means that away < parted < disconnected in regards to color.</p>
<p><img alt="weechat_buflist_formatting screenshot" class="img-responsive" src="https://demu.red/pics/weechat_buflist_formatting.png"/></p>
<h4 id="toning down_1">Toning Down</h4>
<p>As I use <a href="https://weechat.org/scripts/source/screen_away.py.html/">screen_away.py</a> script, and as this buflist.format.buffer overrides hostlist msg colorings, I ended up using this minor tweak to only apply the away to the server buffers (which I have separated).</p>
<div class="highlight"><pre><span></span><code><span class="x">/set buflist.format.buffer "${format_number}${indent}${format_nick_prefix}${color_hotlist}${if:(${buffer.full_name} =~ ^irc)?${if:(${irc_server.away_time} > 0 && ${buffer.name} =~ ^server)?${color:yellow}>}${if:(${irc_channel.part} == 1)?${color:magenta}_}${if:(${irc_server.is_connected} == 0)?${color:*magenta}*}}${if:${type}==private?↪}${name}"</span>
</code></pre></div>
<h5 id="update 2021-02-24: per buffer filter status">Update 2021-02-24: Per buffer filter status</h5>
<p>I have added a bit to check if the <code>/filter</code>'s have been toggled off for individual windows <sup>(Alt+-)</sup> using <code>${if:(${irc_channel.filter} == 0)?${color:red}⨯${color_hotlist}}</code>.
The full setting I use is now:</p>
<div class="highlight"><pre><span></span><code><span class="x">/set buflist.format.buffer "${format_number}${indent}${if:(${buffer.filter} == 0)?${color:magenta}⨯ }${format_nick_prefix}${color_hotlist}${if:(${buffer.full_name} =~ ^irc)?${if:(${irc_server.away_time} > 0 && ${buffer.name} =~ ^server)?${color:yellow}>}${if:(${irc_channel.part} == 1)?${color:magenta}▒}${if:(${irc_server.is_connected} == 0)?${color:*magenta}█}}${if:${type}==private?↪}${name}"</span>
</code></pre></div>
<h3 id="other settings_2">Other Settings</h3>
<p>I'll go ahead and include my other relevant seeming settings for my buflist setup here as well.</p>
<div class="highlight"><pre><span></span><code><span class="x">## copypasta'ed out of /fset</span>
<span class="x">buflist.format.buffer string "${format_number}${indent}${format_nick_prefix}${color_hotlist}${if:(${buffer.full_name} =~ ^irc)?${if:(${irc_server.away_time} > 0)?${color:yellow}>}${if:(${irc_channel.part} == 1)?${color:magenta}_}${if:(${irc_server.is_connected} == 0)?${color:*magenta}*}}${if:${type}==private?↪}${name}"</span>
<span class="x">buflist.format.buffer_current string "${color:red,17}${format_buffer}"</span>
<span class="x">buflist.format.hotlist_highlight string "${color:red}"</span>
<span class="x">buflist.format.hotlist_low string "${color:white}"</span>
<span class="x">buflist.format.hotlist_message string "${color:blue}"</span>
<span class="x">buflist.format.hotlist_none string "${color:247}${if:${type}==server?${color:240}}"</span>
<span class="x">buflist.format.hotlist_private string "${color:red}"</span>
<span class="x">buflist.format.indent string "${color:240}${if:${buffer.next_buffer.local_variables.type}=~^(channel|private)$?├ :└─}"</span>
<span class="x">buflist.format.number string "${color:240}${number}${if:${number_displayed}?.: }"</span>
<span class="x">buflist.look.nick_prefix boolean on</span>
<span class="x">buflist.look.nick_prefix_empty boolean off</span>
</code></pre></div>
<p>The buflist.format.indent is one of the more fancy options, as I like the extra readability.
It should be noted that I have my server buffers split individual, and my buffers hierarchically sorted under each server.
While I used to do this manually, I now use the <a href="https://weechat.org/scripts/source/autosort.py.html/">autosort.py</a> script with some fancy logic to deal with it for me <sup>(and disabled mouse buffer rearrangement to reduce hair pulling)</sup>.</p>
<p>Also worth noting is that I disable the hotlist_low priority msgs from my hotlist/buflist, so I never see it... It mostly just covers content that I would <a href="https://blog.weechat.org/post/2008/10/25/Smart-IRC-join-part-quit-message-filter">smartfilter</a> anyway.</p>
<h3 id="links">Links</h3>
<p><a href="https://weechat.org/files/releasenotes/ReleaseNotes-1.8.html">long ago</a><br/>
<a href="https://weechat.org/">WeeChat</a><br/>
<a href="https://weechat.org/scripts/source/screen_away.py.html/">screen_away.py</a><br/>
<a href="https://weechat.org/scripts/source/autosort.py.html/">autosort.py</a><br/>
<a href="https://blog.weechat.org/post/2008/10/25/Smart-IRC-join-part-quit-message-filter">smartfilter</a> </p>Reduce Oper Server Notice Notification Spam in Android WeeChat Nightly2020-10-30T13:25:00-04:002020-10-30T13:25:00-04:00demuretag:demu.red,2020-10-30:/blog/2020/10/reduce-oper-server-notice-notification-spam-in-android-weechat-nightly/<h3 id="intro">Intro</h3>
<p>After upgrading my phone's OS to android 11, the <a href="https://play.google.com/store/apps/details?id=com.ubergeek42.WeechatAndroid">stable</a> version of Weechat-Android on google play was acting fairly crappy...
Which isn't too surprising considering it was last updated on 2018-10-07 <sup>(and at the time of writing it is 2020-10-30)</sup>.</p>
<p>With some looking, I found that there was a …</p><h3 id="intro">Intro</h3>
<p>After upgrading my phone's OS to android 11, the <a href="https://play.google.com/store/apps/details?id=com.ubergeek42.WeechatAndroid">stable</a> version of Weechat-Android on google play was acting fairly crappy...
Which isn't too surprising considering it was last updated on 2018-10-07 <sup>(and at the time of writing it is 2020-10-30)</sup>.</p>
<p>With some looking, I found that there was a <a href="https://play.google.com/store/apps/details?id=com.ubergeek42.WeechatAndroid.dev">nightly</a> build on google play as well, hurray!
After updating, the WeeChat relay app was acting much better, had a number of improvements, and bug fixes.</p>
<h4 id="so, bug fixes, about that">So, bug fixes, about that</h4>
<p>One of these bug fixes was correcting how ircd service notices where kind of ignored by the app's notification system.
So, say hello to spammy android notices!
At least for those of you who are fellow Opers out there <strong>*waves*</strong></p>
<h3 id="pairing down the spam_1">Pairing down the spam</h3>
<p>Using <code>/debug tags</code> in <a href="https://weechat.org/">WeeChat</a> proper, we can see that a generic connection message has four tags:</p>
<div class="highlight"><pre><span></span><code><span class="x">12:07:36 ~~~ irc.SERVER.org: *** CONNECT: Client connecting on port 6697 (class main): NICK!IDENT@HOST (127.0.0.1) [REALNAME] [irc_notice,notify_private,nick_irc.SERVER.org,host_irc.SERVER.org,log1]</span>
</code></pre></div>
<p>Trimmed down:</p>
<div class="highlight"><pre><span></span><code><span class="x">[irc_notice,notify_private,nick_irc.SERVER.org,host_irc.SERVER.org,log1]</span>
</code></pre></div>
<p>One of these tags, the <code>notify_private</code>, is why we are getting so much spam android notice spam. >_<<br/>
So lets see about setting that to a lower message priority.</p>
<h4 id="the triggers">The Triggers</h4>
<p>Here I came up with two triggers:</p>
<div class="highlight"><pre><span></span><code><span class="x">## Reduce server notice priority on all networks</span>
<span class="x">/trigger addreplace server_notice_priority line "" "${tags} =~ irc_notice" "/notify_private/notify_message/tags"</span>
<span class="x">## Inspircd flavored version to leave XLINE messages (gline/etc) at the higher priority</span>
<span class="x">/trigger addreplace server_notice_priority line "" "${tags} =~ irc_notice && ${message} !~ XLINE:" "/notify_private/notify_message/tags"</span>
</code></pre></div>
<h3 id="afterword_1">Afterword</h3>
<p>With the second version of the trigger, I now will still get a privmsg level indicator in my buflist and hotbar in WeeChat proper -- and an android notification if I'm connected to the relay -- when an XLINE event occurs.</p>
<h3 id="links">Links</h3>
<p><a href="https://play.google.com/store/apps/details?id=com.ubergeek42.WeechatAndroid">stable</a><br/>
<a href="https://play.google.com/store/apps/details?id=com.ubergeek42.WeechatAndroid.dev">nightly</a><br/>
<a href="https://weechat.org/">WeeChat</a> </p>Adding triggers in WeeChat to help parse server notices2020-05-25T16:30:00-04:002020-05-25T16:30:00-04:00demuretag:demu.red,2020-05-25:/blog/2020/05/adding-triggers-in-weechat-to-help-parse-server-notices/<h3 id="intro">Intro</h3>
<p>It's been a while since I've posted, but that's the way of blogs is it not?
This should be a short one.</p>
<p>Anyway, why am I here today?
Because asshats live on the internet :(</p>
<p>Despite having things in place in <a href="https://www.inspircd.org/">InspIRCd</a> like <a href="https://docs.inspircd.org/3/modules/dnsbl/">dnsbl</a> and various other configurations to reduce …</p><h3 id="intro">Intro</h3>
<p>It's been a while since I've posted, but that's the way of blogs is it not?
This should be a short one.</p>
<p>Anyway, why am I here today?
Because asshats live on the internet :(</p>
<p>Despite having things in place in <a href="https://www.inspircd.org/">InspIRCd</a> like <a href="https://docs.inspircd.org/3/modules/dnsbl/">dnsbl</a> and various other configurations to reduce trolls and botnets, people still find ways to ruin things for everyone else.</p>
<p>Some script kiddy decided to use a number of connections from various asian hosts -- many of which were yet to be included on the ip blacklists -- and started spamming.</p>
<h3 id="on to the point">On to the point</h3>
<p>All of this led me to wanting to make the connection/quitting IRC server notices a bit easier to parse.
Bringing us to <a href="https://weechat.org/">WeeChat</a>'s <a href="https://weechat.org/files/doc/stable/weechat_user.en.html#trigger_plugin">triggers</a> and some simple RegEx <a href="https://demu.red/blog/2016/08/dealing-with-facebook-emotes-in-bitlbee-with-weechat/">again</a>.</p>
<h4 id="the triggers">The Triggers</h4>
<p>Here I came up with four triggers to:</p>
<ul>
<li>Highlight the connect msgs and the nick</li>
<li>Highlight if the connect class is "Main" (Inspircd)<ul>
<li>Makes it easy to see that a user is on an external connection</li>
</ul>
</li>
<li>Highlight the quit msgs and the nick</li>
<li>Highlight xline msgs<ul>
<li>Makes it quick to see when dnsbl catches connections</li>
</ul>
</li>
<li><strong>UPDATED:</strong> Highlight announcement/oper msgs</li>
<li><strong>UPDATED:</strong> Added CHANCREATE higlights</li>
</ul>
<p>And the triggers:</p>
<div class="highlight"><pre><span></span><code>/trigger addreplace ircd_notice_con modifier weechat_print <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ irc_notice"</span> <span class="s2">"/(CONNECT:)(.*: )([^ !]+)(!.*)/</span><span class="si">${</span><span class="nv">color</span><span class="p">:</span><span class="nv">02</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">1</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">2</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">14</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">3</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">4</span><span class="si">}</span><span class="s2">/"</span> <span class="s2">""</span>
/trigger addreplace ircd_notice_con_main modifier weechat_print <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ irc_notice"</span> <span class="s2">"/(CONNECT:)(.*class )(main)(.*)/</span><span class="si">${</span><span class="nv">re</span><span class="p">:</span><span class="nv">1</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">2</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">01</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">3</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">4</span><span class="si">}</span><span class="s2">/"</span> <span class="s2">""</span>
/trigger addreplace ircd_notice_quit modifier weechat_print <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ irc_notice"</span> <span class="s2">"/(QUIT:)(.*: )([^ !]+)(!.*)/</span><span class="si">${</span><span class="nv">color</span><span class="p">:</span><span class="nv">03</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">1</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">2</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">06</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">3</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">4</span><span class="si">}</span><span class="s2">/"</span> <span class="s2">""</span>
/trigger addreplace ircd_notice_xline modifier weechat_print <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ irc_notice"</span> <span class="s2">"/(XLINE:)(.*)/</span><span class="si">${</span><span class="nv">color</span><span class="p">:</span><span class="nv">01</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">1</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">2</span><span class="si">}</span><span class="s2">/"</span> <span class="s2">""</span>
/trigger addreplace ircd_notice_announce modifier weechat_print <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ irc_notice"</span> <span class="s2">"/(ANNOUNCEMENT:)(.*)/</span><span class="si">${</span><span class="nv">color</span><span class="p">:</span><span class="nv">05</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">1</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">2</span><span class="si">}</span><span class="s2">/"</span> <span class="s2">""</span>
/trigger addreplace ircd_notice_oper modifier weechat_print <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ irc_notice"</span> <span class="s2">"/(OPER:)(.*)/</span><span class="si">${</span><span class="nv">color</span><span class="p">:</span><span class="nv">05</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">1</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">2</span><span class="si">}</span><span class="s2">/"</span> <span class="s2">""</span>
/trigger addreplace ircd_notice_chancreate modifier weechat_print <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ irc_notice"</span> <span class="s2">"/(CHANCREATE:)(.*Channel )(.*)( created by )(.*)(!.*@.*)/</span><span class="si">${</span><span class="nv">color</span><span class="p">:</span><span class="nv">11</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">1</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">2</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">5</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">3</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">4</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">014</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">5</span><span class="si">}${</span><span class="nv">color</span><span class="p">:</span><span class="nv">reset</span><span class="si">}${</span><span class="nv">re</span><span class="p">:</span><span class="nv">6</span><span class="si">}</span><span class="s2">/"</span> <span class="s2">""</span>
</code></pre></div>
<h3 id="the pudding_1">The Pudding</h3>
<p>For your convenience, here is a screenshot. <sup>(With redactions)</sup>
<img alt="weechat server notice trigger" class="img-responsive" src="https://demu.red/pics/weechat_server_notice_trigger.png"/></p>
<h3 id="links">Links</h3>
<p><a href="https://www.inspircd.org/">InspIRCd</a><br/>
<a href="https://docs.inspircd.org/3/modules/dnsbl/">dnsbl</a><br/>
<a href="https://weechat.org/">WeeChat</a><br/>
<a href="https://weechat.org/files/doc/stable/weechat_user.en.html#trigger_plugin">triggers</a><br/>
<a href="https://demu.red/blog/2016/08/dealing-with-facebook-emotes-in-bitlbee-with-weechat/">again</a> </p>Getting a Yubikey to Work for u2f and as a GPG Smartcard on Void Linux2019-07-25T13:00:00-04:002019-07-25T13:00:00-04:00demuretag:demu.red,2019-07-25:/blog/2019/07/getting-a-yubikey-to-work-for-u2f-and-as-a-gpg-smartcard-on-void-linux/<h3 id="distribution testing">Distribution Testing</h3>
<p>Recently I have been trying a few more linux distributions as I have been on debian sid for a while, but have grown tired of old packages... even on sid.</p>
<p>Since some people I know had been singing <a href="https://voidlinux.org/">Void Linux</a> praises, I figured I might as well try …</p><h3 id="distribution testing">Distribution Testing</h3>
<p>Recently I have been trying a few more linux distributions as I have been on debian sid for a while, but have grown tired of old packages... even on sid.</p>
<p>Since some people I know had been singing <a href="https://voidlinux.org/">Void Linux</a> praises, I figured I might as well try it...
Especially given how often systemd annoys me.</p>
<h3 id="the snafu">The SNAFU</h3>
<p>tl;dr a yubikey doesn't work 'out of the box' on void.</p>
<p>Browsers such as chromium could not do u2f, and gpg only saw the smartcard as root.</p>
<h3 id="fixing things(tm)">Fixing Things™</h3>
<p>I do not claim that this write up is the best way to accomplish setting up a yubikey to work for fido u2f and as a gpg smartcard on void linux.
There may be better ways out there.
There may be a <em>Right Way</em> out there.
But I sure as hell couldn't find a concisely documented way out there...
At least not one that had all the information in one place, seemed current, and verified that void linux would like the solution.</p>
<h4 id="citations">Citations</h4>
<p>Credit where it is due, the arch wiki had a good clear part <sup><a href="https://wiki.archlinux.org/index.php/Chromium/Tips_and_tricks#U2F_authentication">arch wiki reference</a></sup> on getting a yubikey to have uf2 working with browsers such as chromium, and a link to the udev rule <sup><a href="https://github.com/Yubico/libu2f-host/blob/master/70-u2f.rules">70-u2f.rules</a></sup> provided by yubico.
Yubico also already had udev rules <sup><a href="https://github.com/Yubico/yubikey-personalization/blob/master/69-yubikey.rules">69-yubikey.rules</a>, <a href="https://github.com/Yubico/yubikey-personalization/blob/master/70-yubikey.rules">70-yubikey.rules</a></sup> already written up for getting the gpg smartcard to work for non root accounts.</p>
<h4 id="solution">Solution</h4>
<p>So, to get things working we place <a href="https://github.com/Yubico/libu2f-host/blob/master/70-u2f.rules">70-u2f.rules</a>, <a href="https://github.com/Yubico/yubikey-personalization/blob/master/69-yubikey.rules">69-yubikey.rules</a>, and <a href="https://github.com/Yubico/yubikey-personalization/blob/master/70-yubikey.rules">70-yubikey.rules</a> in <code>/etc/udev/rules.d/</code>, and you could give the system a reboot for good measure. <sup>(I dont think the reboot was even needed)</sup></p>
<h3 id="afterword_1">Afterword</h3>
<p>So what is the point of this short blog post?
To remind me how to set this up if I ever need to do so again, and hopefully to bump the answer higher in your search results so you don't spend as much time on this as I had to.</p>
<h3 id="links">Links</h3>
<p><a href="https://voidlinux.org/">Void Linux</a><br/>
<a href="https://wiki.archlinux.org/index.php/Chromium/Tips_and_tricks#U2F_authentication">arch wiki reference</a><br/>
<a href="https://github.com/Yubico/libu2f-host/blob/master/70-u2f.rules">70-u2f.rules</a><br/>
<a href="https://github.com/Yubico/yubikey-personalization/blob/master/69-yubikey.rules">69-yubikey.rules</a><br/>
<a href="https://github.com/Yubico/yubikey-personalization/blob/master/70-yubikey.rules">70-yubikey.rules</a> </p>Endlessh HTML Scoreboard2019-04-26T23:00:00-04:002019-04-26T23:00:00-04:00demuretag:demu.red,2019-04-26:/blog/2019/04/endlessh-html-scoreboard/<h3 id="botnets">Botnets</h3>
<p>As most security concious people know, there are many people out there running automated attacks on the whole of the internet in general.
Among these many attacks are those pointed at the low-hanging-fruit that is <a href="https://www.openssh.com/">OpenSSH</a> on the default port of 22.</p>
<h4 id="basic security">Basic Security</h4>
<p>On that note, if you …</p><h3 id="botnets">Botnets</h3>
<p>As most security concious people know, there are many people out there running automated attacks on the whole of the internet in general.
Among these many attacks are those pointed at the low-hanging-fruit that is <a href="https://www.openssh.com/">OpenSSH</a> on the default port of 22.</p>
<h4 id="basic security">Basic Security</h4>
<p>On that note, if you run a server exposed to the internet you really need to take a few basic security measures:</p>
<ul>
<li>Change your SSH port to a non standard port.</li>
<li>Ensure that root logins are disabled in your <code>sshd_config</code>.<ul>
<li>If root must be enabled, only allow connections with a SSH key.</li>
</ul>
</li>
<li>Use ssh keys whenever possible.<ul>
<li>Disable ssh password login whenever possible.</li>
</ul>
</li>
<li>Use a daemon like <a href="https://www.fail2ban.org/wiki/index.php/Main_Page">Fail2ban</a> or <a href="http://denyhosts.sourceforge.net/">DenyHOSTS</a> to auto ban failed attempts. <sup>(I prefer fail2ban)</sup></li>
</ul>
<h5 id="auto ban daemon">Auto Ban Daemon</h5>
<p>While some may say that using an auto ban daemon is an intermediate/advanced step, if you are running a server exposed to the internet... you need to do it.</p>
<h6 id="denyhosts">DenyHOSTS</h6>
<p><code>denyhosts</code> is an option, and it is fine if it suits your needs.
That being said, it is sooo much more limited.</p>
<h6 id="fail2ban">Fail2ban</h6>
<p><code>fail2ban</code> is amazing.<br/>
tl;dr you pick a text logfile, write a <a href="https://www.regular-expressions.info/">regex</a> to match log lines that are Bad™, and set a ban action.<br/>
The common way this plays out is: read log -> match bad line -> grab ip -> iptables drop ip.</p>
<p>In addition to some of the provided jails you can enable for ssh, I added another jail that matches misc portscan attempts like</p>
<blockquote>
<p>Apr 25 23:45:03 localhost sshd[31130]: Bad protocol version identification 'GET / HTTP/1.1' from 104.168.236.191 port 43262
2019-04-25 23:45:03,796 fail2ban.filter [3963]: INFO [sshd-bad-protocol] Found 104.168.236.191 - 2019-04-25 23:45:03</p>
</blockquote>
<p>since I run on a non standard port. <sup>(line one is auth.log, line two is fail2ban.log)</sup></p>
<h3 id="endlessh_3">endlessh</h3>
<p><a href="https://github.com/skeeto/endlessh">endlessh</a> is a fancy tool written by Chris Wellons.</p>
<blockquote>
<p>Endlessh is an SSH tarpit that very slowly sends an endless, random SSH banner. It keeps SSH clients locked up for hours or even days at a time. The purpose is to put your real SSH server on another port and then let the script kiddies get stuck in this tarpit instead of bothering a real server.</p>
</blockquote>
<p>You can read their write up of <code>endlessh</code> in their <a href="https://nullprogram.com/blog/2019/03/22/">original blog post</a>.</p>
<p>I first heard of <code>endlessh</code> a few weeks ago on in a discussion on IRC, and was immediately taken by the idea.
After getting it set up, I immediately started getting hits in the log.</p>
<h4 id="endlessh setup">endlessh setup</h4>
<p>Since I'm not a big fan of <code>systemd</code> I just added a line to my crontab to run with logging enabled:</p>
<div class="highlight"><pre><span></span><code><span class="x">@reboot root /usr/local/sbin/endlessh -p 22 -v >/var/log/endlessh.log 2>/var/log/endlessh.err</span>
</code></pre></div>
<h3 id="scoreboard_1">Scoreboard</h3>
<p>Not very long after I had <code>endlessh</code> running I had a novel idea....
Lets make a Scoreboard from the log!</p>
<p>Main goals:</p>
<ul>
<li>Use the last seven days.</li>
<li>List connection attempts, min time, max time, average time, and total time of <em>Players</em>.</li>
<li>List geo location of <em>Players</em></li>
</ul>
<h4 id="iteration one: shell script">Iteration One: Shell Script</h4>
<p>For the sheer novelty of it, I decided I would make a shell script just to say I did. <sup>(knowing that I would then need to move to python to add more fancy parts)</sup>
After finishing the shell script to generate an html scoreboard, I quickly moved on to a more feature full python version.</p>
<p>I may have spent a little too much time adding a table sort script into the html...</p>
<h4 id="iteration two: python">Iteration Two: Python</h4>
<p>This iteration went a lot faster as I had already mapped out the objectives and had an idea on the processing.
It was also an interesting comparison to see how much easier it was to do with a real programming language, as I replaced stop gap measures with more robust processing.
While I'm sure there is much that I can improve with my coding, I'm at least pleased that I chunked everything that made sense to be in individual functions. :)</p>
<p>One of the improvements I was able to add after moving to python was the "Top Player" list.</p>
<p>Here you can see my <a href="https://notabug.org/demure/scripts/src/master/endlessh_scoreboard.py">endlessh_scoreboard.py</a>.</p>
<p>Among the future tweaks I think I will end up doing, I think I will have to move from html only to a fancy html5 CSS table.
This is due to the lag that grows and grows the larger the table gets, when attempting to sort.
A change over to CSS (maybe using bootstrap?) would mean more columns of the table should be sortable as well.</p>
<p>As for the point of this post, the Scoreboard itself, you can find that at <strong><a href="http://pit.demu.red/">pit.demu.red</a></strong>.</p>
<h3 id="additional steps_1">Additional Steps</h3>
<p>After all of this there was a little bit of additional system tweaking I did.
Namely adding a <code>fail2ban</code> jail for <em>Players</em> who connect to endlessh too many times <sup>(1000 times in ~five days earns a month ban)</sup>.
And adjusting other <code>fail2ban</code> jails to not block the <em>game's</em> port, which would hobble the <em>game</em>.</p>
<p>fail2ban jail.local:</p>
<div class="highlight"><pre><span></span><code><span class="o">[</span>endlessh<span class="o">]</span>
<span class="nv">enabled</span> <span class="o">=</span> <span class="nb">true</span>
<span class="nv">maxretry</span> <span class="o">=</span> <span class="m">1000</span>
<span class="c1">## One month</span>
<span class="nv">bantime</span> <span class="o">=</span> <span class="m">2419200</span>
<span class="c1">## five days</span>
<span class="nv">findtime</span> <span class="o">=</span> <span class="m">432000</span>
<span class="nv">port</span> <span class="o">=</span> <span class="m">0</span>:65535
<span class="nv">logpath</span> <span class="o">=</span> /var/log/endlessh.log
</code></pre></div>
<p>endlessh.conf (jail):</p>
<div class="highlight"><pre><span></span><code><span class="c1"># fail2ban endlessh ban</span>
<span class="o">[</span>Definition<span class="o">]</span>
<span class="nv">failregex</span> <span class="o">=</span> ^.* ACCEPT <span class="nv">host</span><span class="o">=</span>::ffff:<HOST> <span class="nv">port</span><span class="o">=</span>.*
<span class="c1">#ignoreregex =</span>
</code></pre></div>
<h3 id="future goals">Future Goals</h3>
<p>Besides a revision to move to a CSS table with sorting, I may end up adding some code to generate a ascii table for gopher.</p>
<h3 id="links">Links</h3>
<p><a href="https://www.openssh.com/">OpenSSH</a><br/>
<a href="https://www.fail2ban.org/wiki/index.php/Main_Page">Fail2ban</a><br/>
<a href="http://denyhosts.sourceforge.net/">DenyHOSTS</a><br/>
<a href="https://www.regular-expressions.info/">regex</a><br/>
<a href="https://github.com/skeeto/endlessh">endlessh</a><br/>
<a href="https://nullprogram.com/blog/2019/03/22/">original blog post</a><br/>
<a href="https://notabug.org/demure/scripts/src/master/endlessh_scoreboard.py">endlessh_scoreboard.py</a><br/>
<a href="http://pit.demu.red/">pit.demu.red</a> </p>NeoMutt Macro - Opening text/html attachment from the index view2017-11-29T22:00:00-05:002017-11-29T22:00:00-05:00demuretag:demu.red,2017-11-29:/blog/2017/11/neomutt-macro-opening-texthtml-attachment-from-the-index-view/<h3 id="neomutt">NeoMutt</h3>
<p><a href="https://www.neomutt.org/">NeoMutt</a> is a fork of <a href="http://www.mutt.org/">mutt</a> which mainlines a vast number of needed patches <sup>(while removing the headache of having them)</sup>, and actually has ongoing development.</p>
<blockquote>
<p>What is it?<br/>
NeoMutt is a command line mail reader (or MUA). It's a version of Mutt with added features.</p>
<p>Why?<br/>
The NeoMutt …</p></blockquote><h3 id="neomutt">NeoMutt</h3>
<p><a href="https://www.neomutt.org/">NeoMutt</a> is a fork of <a href="http://www.mutt.org/">mutt</a> which mainlines a vast number of needed patches <sup>(while removing the headache of having them)</sup>, and actually has ongoing development.</p>
<blockquote>
<p>What is it?<br/>
NeoMutt is a command line mail reader (or MUA). It's a version of Mutt with added features.</p>
<p>Why?<br/>
The NeoMutt project is hoping to kick-start development on the Mutt project.<br/>
NeoMutt has already attracted about twenty developers and enthusiasts. </p>
<p>What have you achieved, so far?<br/>
Lots of old Mutt patches have been brought up-to-date, tidied and documented.<br/>
Notably, the Sidebar patch has now been adopted by upstream Mutt. </p>
</blockquote>
<p>I've been using neomutt for quite some time, and it is a great improvement over the original mutt.</p>
<h3 id="too many emails [that use html]">Too Many Emails [that use html]</h3>
<p>Sadly, every day I get five to twenty html only emails that are worth enough of my time that I haven't unsubscribed, and don't delete on sight.
Of these emails, it feels like 98% are the kind that have no meaningful content in text only view :/ ...</p>
<h4 id="text only html">Text Only HTML</h4>
<p>Auto viewing html emails as text is a combination of a couple of settings.
You can read up on some of it on neomutt's <a href="https://www.neomutt.org/guide/mimesupport">mimesupport</a> page.</p>
<p><strong>muttrc</strong></p>
<div class="highlight"><pre><span></span><code><span class="x">auto_view text/html</span>
<span class="x">alternative_order text/enriched text/plain text/html text</span>
</code></pre></div>
<p><strong>mailcap</strong></p>
<div class="highlight"><pre><span></span><code><span class="x">text/html; ~/.mutt/view_attachment.sh %s html; test=test -n "$DISPLAY"</span>
<span class="x">text/html; w3m %s; nametemplate=%s.html; needsterminal</span>
<span class="x">text/html; w3m -v -F -T text/html -dump %s; copiousoutput</span>
</code></pre></div>
<p>Where <code>auto_view</code> uses the third mailcap entry due to <code>copiousoutput</code>.</p>
<h4 id="ugg... image based html">Ugg... Image Based HTML</h4>
<p>So as I was saying, too many of these emails are gobbledygook unless viewed with images rendered.
This means I am in the habit of pressing a number of keys in neomutt to launch them in a browser.</p>
<div class="highlight"><pre><span></span><code><span class="x"><enter> ## Open the email, and see that it is illegible</span>
<span class="x">v ## Open attachment view</span>
<span class="x"><down><down> ## Select the text/html attachment</span>
<span class="x"><enter> ## Open the attachment</span>
<span class="x"><super_L> 3 ## Have i3 jump to qutebrowser's workspace</span>
</code></pre></div>
<p>Sure, the last one isn't neomutt's (and I will still have it regardless), but that is approximately four to five keystrokes to do-one-thing...</p>
<h3 id="black friday_1">Black Friday</h3>
<p>This <s>spendsgiving</s> thanksgiving I got at least 100 emails trying to get me to spend money, and I thought about how I wanted a better way to open the html emails from sources I at least wanted to glance at.</p>
<h3 id="...cyber monday">...Cyber Monday</h3>
<p>And then cyber monday rolled around and I got another 300 emails <sup>(I may be exaggerating)</sup>.</p>
<h3 id="the macro">The Macro</h3>
<p>Initially I was overthinking the issue, and trying to find some fancy answer.
My thanks to the people on freenode's #neomutt for humoring me (flatcap, antonio_).
After stepping a bit back from the issue and looking at my other macros, I think I have a fairly succinct answer which should work in most situations:</p>
<p><strong>muttrc</strong></p>
<div class="highlight"><pre><span></span><code><span class="x">## Make quick html view macro</span>
<span class="x">bind index,pager V noop ## Unbinds V from version</span>
<span class="x">macro index,pager V "<view-attachments><search>html<enter><view-mailcap><exit>"</span>
</code></pre></div>
<p>where <code><view-attachments></code> opens the attachment view, <code><search>html<enter></code> searchs for 'html' and selects the result, and <code><view-mailcap><exit></code> opens the text/html attachment with the mailcap and returns to the starting view.</p>
<h4 id="notes">Notes</h4>
<ul>
<li>This would need a line like <code>text/html; ~/.mutt/view_attachment.sh %s html; test=test -n "$DISPLAY"</code> in your mailcap.</li>
<li>This assumes that there is only one html mime type in the file. I haven't run into more than one.</li>
</ul>
<h3 id="the results_1">The Results</h3>
<p>I get to press less keys <sup>(one, plus a workspace switch)</sup> when I read my annoying html emails for of images!</p>
<h3 id="links">Links</h3>
<p><a href="https://www.neomutt.org/">NeoMutt</a><br/>
<a href="http://www.mutt.org/">mutt</a><br/>
<a href="https://www.neomutt.org/guide/mimesupport">mimesupport</a> </p>AWK is great - fail2ban list2017-09-23T14:00:00-04:002017-09-23T14:00:00-04:00demuretag:demu.red,2017-09-23:/blog/2017/09/awk-is-great-fail2ban-list/<h3 id="all hail awk">All hail AWK</h3>
<p>It took me a while to get around to teaching myself <a href="http://www.grymoire.com/Unix/Awk.html">awk</a>, but since learning it some years ago I have not looked back.
There is quite a lot that you can do with it; while sometimes a full program (python/ruby/etc) might be more robust …</p><h3 id="all hail awk">All hail AWK</h3>
<p>It took me a while to get around to teaching myself <a href="http://www.grymoire.com/Unix/Awk.html">awk</a>, but since learning it some years ago I have not looked back.
There is quite a lot that you can do with it; while sometimes a full program (python/ruby/etc) might be more robust...
There is something very satisfying about perfecting an awk 'oneliner'.
<sup>(I may stretch some peoples' definition of oneliner)</sup>
While I say 'awk' in this article, I use <a href="https://www.gnu.org/software/gawk/manual/gawk.html">gawk</a> as it has some nice things added.</p>
<h4 id="regex is also great">regex is also great</h4>
<p>If you haven't taken the time to learn <a href="https://www.gnu.org/software/grep/manual/html_node/Regular-Expressions.html">regex</a>, and some of it's more advanced features, you are missing out.
It's flexiblity to handle different matches is amazing, and once you start using <a href="http://www.rexegg.com/regex-capture.html">capture groups</a> you will wonder why no one told you sooner.</p>
<h5 id="minor variations on a theme">Minor variations on a theme</h5>
<p>You will want to note that there is a minor bit of variation between implementations of regex.
<sup>(though nowhere near as bad as markdown fragmentation)</sup>
This mostly consists newer implementations having a few more features bolted on, that are followed in other yet newer implementations -- you just will miss them in older programs like <code>sed</code>.
One example would be what is referred to as <code>perl regex</code>, which adds some nice escape codes like: <code>\d</code>, <code>\D</code>, <code>\w</code>, <code>\W</code>, <code>\s</code>, <code>\S</code>, etc.</p>
<p>Here <code>\d</code> is a quick way to specify digit, though you can always use the old school way instead: <code>[0-9]</code>.
With other added escape codes the old school way gets more tedious, so being able to use <code>\w</code> for word character is very convenient -- and <code>\W</code> for any NOT word character.</p>
<h3 id="today's sample_2">Today's sample</h3>
<p>Niro, a friend on the interwebz, was asking for a convenient way of listing all IP's currently blocked by <a href="https://www.fail2ban.org/">fail2ban</a> while only listing a unique IP a single time.</p>
<p>My initial google fu <a href="https://askubuntu.com/a/893108">turned up</a>:</p>
<div class="highlight"><pre><span></span><code>fail2ban-client status <span class="p">|</span> grep <span class="s2">"Jail list:"</span> <span class="p">|</span> sed <span class="s2">"s/ //g"</span> <span class="p">|</span> awk <span class="s1">'{split($2,a,",");for(i in a) system("fail2ban-client status " a[i])}'</span> <span class="p">|</span> grep <span class="s2">"Status\|IP list"</span>
</code></pre></div>
<p>This did list each jail's banned IPs...
Though it also printed cruff for jails without current bans, cruff around the IPs, and didn't limit output to unique IPs.
It also failed to use awk to its full potential. *<strong>points at the <code>grep</code> to <code>sed</code> to <code>awk</code> to... <code>grep</code></strong>*</p>
<h4 id="awk remix">awk remix</h4>
<p>So, lets turn that awk up to 11:</p>
<div class="highlight"><pre><span></span><code>fail2ban-client status <span class="p">|</span> awk <span class="s1">'/Jail list/ {match($0, /^.*\:\s+(.*)/, m1); split(m1[1], s, ", "); for(i in s){while("fail2ban-client status " s[i] |& getline var){match(var, /IP list\:\s+(.*)/, m2); if(m2[1] != ""){h=h m2[1]}}}; gsub(/ /, "\n", h); print h | "sort -u"}'</span>
</code></pre></div>
<p><strong>NOTE:</strong> This is using <code>gawk</code> for the fancy <code>match(){}</code> function.</p>
<p>Here is the same thing broken up into a more readable format:</p>
<div class="highlight"><pre><span></span><code><span class="x">fail2ban-client status | ## Get raw list of jails</span>
<span class="x"> awk '/Jail list/ ## Get single relivant line</span>
<span class="x"> {</span>
<span class="x"> match($0, /^.*\:\s+(.*)/, m1); ## Grab just the jails</span>
<span class="x"> split(m1[1], s, ", "); ## Make jails into usable format</span>
<span class="x"> for(i in s) ## Iterate over jails</span>
<span class="x"> {</span>
<span class="x"> while("fail2ban-client status " s[i] |& getline var) ## Get raw list of IPs for each jail</span>
<span class="x"> {</span>
<span class="x"> match(var, /IP list\:\s+(.*)/, m2); ## Grab just the IPs for each jail</span>
<span class="x"> if(m2[1] != "") ## Weed out non matches</span>
<span class="x"> {</span>
<span class="x"> h=h m2[1] ## Append and merge IPs to list</span>
<span class="x"> }</span>
<span class="x"> }</span>
<span class="x"> };</span>
<span class="x"> gsub(/ /, "\n", h); ## Separate IPs with newline</span>
<span class="x"> print h | "sort -u" ## sort, uniq, and print IPs one per line</span>
<span class="x"> }'</span>
</code></pre></div>
<h3 id="tldr;_1">TLDR;</h3>
<p>You should learn awk and regex. :)</p>
<h3 id="links">Links</h3>
<p><a href="http://www.grymoire.com/Unix/Awk.html">awk</a><br/>
<a href="https://www.gnu.org/software/gawk/manual/gawk.html">gawk</a><br/>
<a href="https://www.gnu.org/software/grep/manual/html_node/Regular-Expressions.html">regex</a><br/>
<a href="http://www.rexegg.com/regex-capture.html">capture groups</a><br/>
<a href="https://www.fail2ban.org/">fail2ban</a> </p>How to check if your smartcard's GPG key is in cache PART 32017-03-13T17:05:00-04:002017-03-13T17:05:00-04:00demuretag:demu.red,2017-03-13:/blog/2017/03/how-to-check-if-your-smartcards-gpg-key-is-in-cache-part-3/<h3 id="gpg smartcard cache checking">GPG SmartCard Cache Checking</h3>
<p>In previous posts I talked about checking if a gpg key was cached, and used a broad check with my awk to do so.
By 'broad', I just wish to point out that you <em>could</em> build the check around a specific key-grip if you cared to …</p><h3 id="gpg smartcard cache checking">GPG SmartCard Cache Checking</h3>
<p>In previous posts I talked about checking if a gpg key was cached, and used a broad check with my awk to do so.
By 'broad', I just wish to point out that you <em>could</em> build the check around a specific key-grip if you cared to.
And recently I got around to ordering a 'traditional' smartcard from <a href="http://shop.kernelconcepts.de/">kernel concepts</a>.
While I've had and used a <a href="https://www.yubico.com/products/yubikey-hardware/yubikey-neo/">yubikey neo</a> for a while with my laptop/phone, I have been utilizing an on laptop subkey for 'convenience'.
The local subkey was handling my <code>pass</code> (<a href="https://www.passwordstore.org/">password store</a>) datebase, and in the past my <code>cryptshotr</code> keyfile.</p>
<h4 id="personal desires">Personal Desires</h4>
<p>My reason for getting a classic style smartcard are as follows:</p>
<ol>
<li>I have the card slot on my laptop.</li>
<li>I don't want to have a fullsized yubikey sticking out all the time.</li>
<li>I don't want to burn a USB port by always having a yubikey nano living in it.</li>
</ol>
<h3 id="hands on_1">Hands on</h3>
<p>After getting the card I trimmed the edge down so that it barely sticks out from the laptop.</p>
<h4 id="multiple smartcard woes">Multiple SmartCard Woes</h4>
<p>To set up my new card, I pulled a copy of my non-stub laptop keys (encryption, signature, and authentication) out of secure storage, and ran a <code>export GNUPGHOME=...</code> to point at the copy.
Once my keys and configurations are finished on the new card and I was back to my normal keyring <sup>(temp dir having been shreaded)</sup>, I ran into an issue where it kept asking for the yubikey by serial...
Unfortunately running <code>gpg --card-edit</code> and <code>fetch</code> did not help.
To fix this I ran good old <code>gpg-connect-agent 'keyinfo --list' /bye 2>/dev/null</code> <sup>(from the previous posts: <a href="https://demu.red/blog/2016/06/how-to-check-if-your-gpg-key-is-in-cache/">Part1</a>, <a href="https://demu.red/blog/2016/08/how-to-check-if-your-gpg-key-is-in-cache-part-2/">Part2</a>)</sup> to identify the key-grips, and deleted the three offending <code>/.gnupg/private-keys-v1.d/KEY_GRIP_HERE.key</code> files.
After they were cleared out, I was successfully about to <code>gpg --card-edit</code> and <code>fetch</code> from the new key.</p>
<h4 id="verifying functionality">Verifying Functionality</h4>
<p>While testing that the keys on the smartcard worked, I also checked on my previous cache check solution.
Sadly I found that <code>gpg-connect-agent 'keyinfo --list' /bye 2>/dev/null</code> doesn't show the smartcard status.</p>
<p>While reading thought the output of <code>gpg-connect-agent 'help' /bye</code>, and each individual help page, I see <code>scd</code>.</p>
<div class="highlight"><pre><span></span><code>~ -> gpg-connect-agent <span class="s1">'help scd'</span> /bye
<span class="c1"># SCD <commands to pass to the scdaemon></span>
<span class="c1">#</span>
<span class="c1"># This is a general quote command to redirect everything to the</span>
<span class="c1"># SCdaemon.</span>
OK
</code></pre></div>
<h4 id="alice falling down the rabbit hole">Alice Falling Down the Rabbit Hole</h4>
<p>So now, here I am in my <code>bash</code> shell, sending commands to the <code>gpg-agent</code> shell, sending commands to the <code>scdaemon</code> shell.</p>
<div class="highlight"><pre><span></span><code>~ -> gpg-connect-agent <span class="s1">'scd help'</span> /bye
<span class="c1"># NOP</span>
<span class="c1"># CANCEL</span>
<span class="c1"># OPTION</span>
<span class="c1"># BYE</span>
<span class="c1"># AUTH</span>
<span class="c1"># RESET</span>
<span class="c1"># END</span>
<span class="c1"># HELP</span>
<span class="c1"># SERIALNO [--demand=<serialno>] [<apptype>]</span>
<span class="c1"># LEARN [--force] [--keypairinfo]</span>
<span class="c1"># READCERT <hexified_certid>|<keyid></span>
<span class="c1"># READKEY [--advanced] <keyid></span>
<span class="c1"># SETDATA [--append] <hexstring></span>
<span class="c1"># PKSIGN [--hash=[rmd160|sha{1,224,256,384,512}|md5]] <hexified_id></span>
<span class="c1"># PKAUTH <hexified_id></span>
<span class="c1"># PKDECRYPT <hexified_id></span>
<span class="c1"># INPUT</span>
<span class="c1"># OUTPUT</span>
<span class="c1"># GETATTR <name></span>
<span class="c1"># SETATTR <name> <value></span>
<span class="c1"># WRITECERT <hexified_certid></span>
<span class="c1"># WRITEKEY [--force] <keyid></span>
<span class="c1"># GENKEY [--force] [--timestamp=<isodate>] <no></span>
<span class="c1"># RANDOM <nbytes></span>
<span class="c1"># PASSWD [--reset] [--nullpin] <chvno></span>
<span class="c1"># CHECKPIN <idstr></span>
<span class="c1"># LOCK [--wait]</span>
<span class="c1"># UNLOCK</span>
<span class="c1"># GETINFO <what></span>
<span class="c1"># RESTART</span>
<span class="c1"># DISCONNECT</span>
<span class="c1"># APDU [--[dump-]atr] [--more] [--exlen[=N]] [hexstring]</span>
<span class="c1"># KILLSCD</span>
OK
</code></pre></div>
<p>While reading through the help pages on each of these, <code>GETINFO</code> looks promising.</p>
<div class="highlight"><pre><span></span><code>~ -> gpg-connect-agent <span class="s1">'scd help getinfo'</span> /bye
<span class="c1"># GETINFO <what></span>
<span class="c1">#</span>
<span class="c1"># Multi purpose command to return certain information.</span>
<span class="c1"># Supported values of WHAT are:</span>
<span class="c1">#</span>
<span class="c1"># version - Return the version of the program.</span>
<span class="c1"># pid - Return the process id of the server.</span>
<span class="c1">#</span>
<span class="c1"># socket_name - Return the name of the socket.</span>
<span class="c1">#</span>
<span class="c1"># status - Return the status of the current reader (in the future, may</span>
<span class="c1"># also return the status of all readers). The status is a list of</span>
<span class="c1"># one-character flags. The following flags are currently defined:</span>
<span class="c1"># 'u' Usable card present. This is the normal state during operation.</span>
<span class="c1"># 'r' Card removed. A reset is necessary.</span>
<span class="c1"># These flags are exclusive.</span>
<span class="c1">#</span>
<span class="c1"># reader_list - Return a list of detected card readers. Does</span>
<span class="c1"># currently only work with the internal CCID driver.</span>
<span class="c1">#</span>
<span class="c1"># deny_admin - Returns OK if admin commands are not allowed or</span>
<span class="c1"># GPG_ERR_GENERAL if admin commands are allowed.</span>
<span class="c1">#</span>
<span class="c1"># app_list - Return a list of supported applications. One</span>
<span class="c1"># application per line, fields delimited by colons,</span>
<span class="c1"># first field is the name.</span>
<span class="c1">#</span>
<span class="c1"># card_list - Return a list of serial numbers of active cards,</span>
<span class="c1"># using a status response.</span>
OK
</code></pre></div>
<p>Particularly, I noted <code>card_list</code> and its used of the word <em>'active'</em>.</p>
<p>Upon testing, I find that <code>gpg-connect-agent 'scd getinfo card_list' /bye</code> is a seemingly reliable way of seeing if a smartcard key is in cache.</p>
<p>If the card is inserted <strong>AND</strong> the card has been used to decrypt you get:</p>
<div class="highlight"><pre><span></span><code>~ -> gpg-connect-agent <span class="s1">'scd getinfo card_list'</span> /bye
S SERIALNO REDACTEDREDACTEDREDACTEDREDACTED
OK
</code></pre></div>
<p>If the card is removed, if <code>scdaemon</code> wasn't running, or if the card is inserted <strong>AND</strong> the card has not been used:</p>
<div class="highlight"><pre><span></span><code>~ -> gpg-connect-agent <span class="s1">'scd getinfo card_list'</span> /bye
OK
</code></pre></div>
<h3 id="implementing_1">Implementing</h3>
<p>The following is a tasty little piece of awk that check is any smartcard is listed as active, and returns '<code>1</code>' if yes.</p>
<div class="highlight"><pre><span></span><code>gpg-connect-agent <span class="s1">'scd getinfo card_list'</span> /bye <span class="m">2</span>>/dev/null <span class="p">|</span> awk <span class="s1">'BEGIN {CH=0} /SERIALNO/ {if($0!=""){CH=1}} END {print CH}'</span>
</code></pre></div>
<p>And for a cherry on top, there is a <em>'oneliner'</em> that will check if either a local key or a smartcard is cached:</p>
<div class="highlight"><pre><span></span><code><span class="o">{</span> gpg-connect-agent <span class="s1">'keyinfo --list'</span> /bye <span class="m">2</span>>/dev/null<span class="p">;</span> gpg-connect-agent <span class="s1">'scd getinfo card_list'</span> /bye <span class="m">2</span>>/dev/null<span class="p">;</span> <span class="o">}</span> <span class="p">|</span> awk <span class="s1">'BEGIN{CH=0} /^S/ {if($7==1){CH=1}; if($2=="SERIALNO"){CH=1}} END{if($0!=""){print CH} else {print "none"}}'</span>
</code></pre></div>
<h3 id="bonus: pictures of smartcards in a thinkpad.">Bonus: Pictures of smartcards in a Thinkpad.</h3>
<p>Because it is <strong><em>impossible</em></strong> to find a picture of a smartcard in a Thinkpad laptop on the internet, here is my x260 with a card in it.</p>
<ul>
<li>normal
<img alt="thinkpad_smartcard_full" class="img-responsive" src="https://demu.red/pics/thinkpad_smartcard_full.jpg"/></li>
<li>trimmed
<img alt="thinkpad_smartcard_cut" class="img-responsive" src="https://demu.red/pics/thinkpad_smartcard_cut.jpg"/></li>
</ul>
<h3 id="links">Links</h3>
<p><a href="http://shop.kernelconcepts.de/">kernel concepts</a><br/>
<a href="https://www.yubico.com/products/yubikey-hardware/yubikey-neo/">yubikey neo</a><br/>
<a href="https://www.passwordstore.org/">password store</a><br/>
<a href="https://demu.red/blog/2016/06/how-to-check-if-your-gpg-key-is-in-cache/">Part1</a><br/>
<a href="https://demu.red/blog/2016/08/how-to-check-if-your-gpg-key-is-in-cache-part-2/">Part2</a> </p>Backups -- Revisited2017-02-09T04:59:00-05:002017-02-09T04:59:00-05:00demuretag:demu.red,2017-02-09:/blog/2017/02/backups-revisited/<h3 id="backups">Backups</h3>
<blockquote>
<p>blah blah blah.
We all know, or should know, that backups are important.</p>
</blockquote>
<p>As I mentioned in <a href="https://demu.red/articles/008-backups_conky_display.md">my first post about backups</a>, they are important.
I could link to so many statistics about lost data here... but then I would need to vet their legitimacy. *<em>wink</em>*</p>
<h3 id="options">Options</h3>
<p>Again, as …</p><h3 id="backups">Backups</h3>
<blockquote>
<p>blah blah blah.
We all know, or should know, that backups are important.</p>
</blockquote>
<p>As I mentioned in <a href="https://demu.red/articles/008-backups_conky_display.md">my first post about backups</a>, they are important.
I could link to so many statistics about lost data here... but then I would need to vet their legitimacy. *<em>wink</em>*</p>
<h3 id="options">Options</h3>
<p>Again, as I mentioned last time:</p>
<blockquote>
<p>There are a <a href="https://wiki.archlinux.org/index.php/Synchronization_and_backup_programs">Lot of Options</a> out there.
A lot of them don't do what I want.
There are a few criteria that I desire, deltas, encrypted transfer, encrypted storage, space conservation, remote backup.
There are few programs that did these, and did them in a way I liked.</p>
</blockquote>
<p>Last time I talked about <code>rsnapshot</code> and <code>obnam</code>.
After close to a year of using my <a href="https://notabug.org/demure/cryptshotr/src/master/cryptshotr">cryptshotr</a> (a fork of <a href="https://github.com/pigmonkey/backups/blob/master/cryptshot.sh">cryptshot</a>), a <code>rsnapshot</code> wrapper, I have learned more about what I want.
I have also gotten myself a shiny NAS recently.</p>
<h4 id="nas">NAS</h4>
<p>I got a QNAP TS-453A recently, and put four 4TB WesternDigital Red Drives in it.
Sadly, one drive was dead on arrival, and I had to go through warranty.</p>
<p>After getting the replacement drive, I now have a RAID6 with ~7.1TB of usable space!
<sup>(About two terabytes were immediately filled with content)</sup></p>
<p>Having gotten this nice, big, directly networked, high network speed/disk speed beast, I decided it was time to relook at my backups.</p>
<h4 id="rsnapshot">rsnapshot</h4>
<p>My <a href="http://rsnapshot.org/">rsnapshot</a> based backups have worked, and were reliable, but had two 'issues'.
They where slower than I would like. An incremental backup would take about 15-20 minutes.
And they use an encrypted <code>LUKS</code>+<code>dmcrypt</code>, which isn't going to fly with my NAS.</p>
<p>Now, I'm sure that a big part of the time sink was needing to decrypt/mount the USB 3 external for every backup. <sup>(And the pi3 only has so much CPU...)</sup></p>
<p>But to be honest, <code>cryptshotr</code> is also a bit <a href="https://en.wikipedia.org/wiki/Rube_Goldberg_machine">Rube Goldberg-esk</a>.
<img alt="cryptshotr diagram" class="img-responsive" src="https://demu.red/pics/backups-cryptshotr.png"/></p>
<h4 id="attic">Attic</h4>
<p>After yet another round of backup option research, <a href="https://github.com/jborg/attic">Attic</a> was looking like a decent option.</p>
<ul>
<li>works over the network</li>
<li>has built in encryption</li>
<li>does deltas</li>
<li>does de-duplication</li>
</ul>
<p>However when I went to install in on debian sid, I found it was no longer in the repo.
With a bit more digging, I notices it was looking a bit <em>stale</em>.
While looking at the git issues, I noticed <a href="https://twitter.com/thomasjwaldmann?lang=en">ThomasWaldmann</a> referencing issues to <a href="https://github.com/borgbackup/borg">BorgBackup</a>, so I gave it a peek.</p>
<h3 id="borgbackup (aka borg)_1">BorgBackup (aka borg)</h3>
<h4 id="what are the differences between attic and borg?"><a href="https://borgbackup.readthedocs.io/en/stable/faq.html#what-are-the-differences-between-attic-and-borg">What are the differences between Attic and Borg?</a></h4>
<blockquote>
<p>Borg is a fork of Attic and maintained by “The Borg collective”.</p>
<p>Here’s a (incomplete) list of some major changes:</p>
<ul>
<li>more open, faster paced development (see issue #1)</li>
<li>lots of attic issues fixed (see issue #5)</li>
<li>less chunk management overhead (less memory and disk usage for chunks index)</li>
<li>faster remote cache resync (useful when backing up multiple machines into same repo)</li>
<li>compression: no, lz4, zlib or lzma compression, adjustable compression levels</li>
<li>repokey replaces problematic passphrase mode (you can’t change the passphrase nor the pbkdf2 iteration count in “passphrase” mode)</li>
<li>simple sparse file support, great for virtual machine disk files</li>
<li>can read special files (e.g. block devices) or from stdin, write to stdout</li>
<li>mkdir-based locking is more compatible than attic’s posix locking</li>
<li>uses fadvise to not spoil / blow up the fs cache</li>
<li>better error messages / exception handling</li>
<li>better logging, screen output, progress indication</li>
<li>tested on misc. Linux systems, 32 and 64bit, FreeBSD, OpenBSD, NetBSD, Mac OS X</li>
</ul>
</blockquote>
<h4 id="usage">Usage</h4>
<p>Usage is pretty simple. To initialize a backup location <code>borg init /path/to/repo</code> and to make a backup <code>borg create /path/to/repo::archive_name_x ~/one ~/two</code>.
Then you can start dropping options on top of that.</p>
<h4 id="autofs">AutoFS</h4>
<p>Initially I was pointing my backups at an <a href="https://wiki.archlinux.org/index.php/autofs">AutoFS</a> local <code>CIFS</code> mount, which worked quite well.
*<em>skims past</em>*
After getting everything working and automated <sup>(more on that later)</sup>, I decided to switch over to the ssh connection type so I could start backing up my VPS too.<sup>(Since... uh... I'm not going to be doing a CIFS mount over the Internet)</sup></p>
<p><strong><em>I should also note that incremental updates are only taking one to three minutes now!!! (with ~45gb of space that <code>borg</code> is paying attention to).</em></strong></p>
<h3 id="setting up ssh_1">Setting up SSH</h3>
<p>This proved to be a big of a chore.
With my QNAP I have a third party package called <a href="https://github.com/Entware-ng/Entware-ng">Entware-ng</a> install, which provides the <code>opkg</code> package manager from <a href="https://openwrt.org/">OpenWRT</a> (and I have used on my router).
This is a handy way to get various linux packages onto the NAS, like my <code>deluged</code>.
Unfortunately <code>borg</code> wasn't an offered package.
The precompiled binaries also have <a href="https://github.com/borgbackup/borg/issues/690">issues</a>.</p>
<h4 id="qnap container">QNAP Container</h4>
<p>This seemed pretty easy.
Install the app called 'Container Station', and wait for it to install itself and its dependency apps.
Then make a container of your chosen linux distro <sup>(please don't say 'ubuntu'...)</sup>.
I choose a debian lxc <sup>(which I promptly upgraded to debian sid)</sup>.
During the setup process, select 'Advanced Settings>>>', set up the Port Forwarding, and the Shared Folders. <sup>(I didn't do this the first time round...)</sup></p>
<h5 id="fixing qnap container port forwarding">Fixing QNAP Container Port Forwarding</h5>
<p>So... about Port Forwarding...
The UI only lets you edit it during container creation...</p>
<p>Bug anyone?</p>
<p>At this point I start using my GoogleFu, and see that a few people have posted about this over the past couple of years... with no solutions known.</p>
<p>Oooo the bug gets funner! After restarting my container, the Port Forward vanished!</p>
<p>Well, I decide it can't hurt to start digging though the files on the filesystem, and I luck out.
Behold the <code>qnap.json</code>, found at <code>/share/ROOT_OF_RAID/SHARE_NAME/container-station-data/lib/lxc/NAME_OF_CONTAINER</code>.
In my case: <code>/share/CACHEDEV1_DATA/containers/container-station-data/lib/lxc/debian</code>.</p>
<p>Reading the now <em>'bad'</em> file, I find:</p>
<div class="highlight"><pre><span></span><code><span class="x">{"volume": {"new": [], "host": {}, "container": []}, "version": "8", "resource": {"device": [], "limit": {}}, "name": "bad-container", "autostart": true, "arch": "amd64", "image": "debian-jessie", "type": "lxc", "network": {"hostname": "bad-container", "port": [], "mode": "nat"}}</span>
</code></pre></div>
<p>After making a through away container, I find a good one should look like:</p>
<div class="highlight"><pre><span></span><code><span class="x">{"volume": {"new": [], "host": {}, "container": []}, "version": "8", "resource": {"device": [], "limit": {}}, "name": "throw-away", "autostart": true, "arch": "amd64", "image": "debian-jessie", "type": "lxc", "network": {"hostname": "throw-away", "port": [[1111, 22, "TCP"]], "mode": "nat"}}</span>
</code></pre></div>
<p>Where <code>[[1111, 22, "TCP"]]</code> is <code>[[Exterior_port, container_port, "TCP_or_UDP"]]</code>.
While I did not test, I suspect multiple ports would look like <code>[[1111, 22, "TCP"], [2222, 8080, "TCP"], [ETC...]]</code></p>
<p>Which is now <a href="https://forum.qnap.com/viewtopic.php?t=115258#p596749">posted</a> on the QNAP forum as well.</p>
<h3 id="end setup_2">End Setup</h3>
<p>Here is a summary of how I tied everything together.</p>
<h4 id="borg_wrap.sh">borg_wrap.sh</h4>
<p>I wrote up a nice little script to handle my backups.</p>
<p>It features:</p>
<ul>
<li>Error handling</li>
<li>Config items, and config checking</li>
<li>Tests that I am on home network</li>
<li><code>pass</code> (<a href="https://www.passwordstore.org/">passwordstore</a>) is used to store the password<ul>
<li>Relies on gpg key being cached to access password</li>
</ul>
</li>
</ul>
<p><a href="https://notabug.org/demure/scripts/src/master/borg_wrap.sh">borg_wrap.sh code</a></p>
<h4 id="users">Users</h4>
<p>Users? We don't need this section!
Well, aside from a normal account on the destination with access to the backup destination.</p>
<h4 id="ssh">SSH</h4>
<div class="highlight"><pre><span></span><code>ssh-ed25519 XXX... laptop_key
<span class="nv">command</span><span class="o">=</span><span class="s2">"borg serve --restrict-to-path /mnt/borg/laptop"</span>,no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-user-rc ssh-ed25519 XXX...
<span class="nv">command</span><span class="o">=</span><span class="s2">"borg serve --restrict-to-path /mnt/borg/server"</span>,no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-user-rc ssh-ed25519 XXX...
</code></pre></div>
<p>Notable parts:</p>
<ul>
<li>The script calls a passphraseless key, so that it doesn't need to source ssh-agent.<ul>
<li>Trying to source ssh-agent with crontab is possible, but sudo just makes that a painful thought.</li>
</ul>
</li>
<li>The <code>borg serve --restirct-to-path</code> both limits borg serve's access, and prevents non-matching commands from working -- thus preventing unauthorized access.</li>
<li>Due to QNAP's... Interesting Port Forwarding set up, I can't limit the keys with <code>from=</code>, as it isn't seen correctly by the container.</li>
</ul>
<h4 id="sudoers.d">Sudoers.d</h4>
<p>Here is the content of my <code>/etc/sudoers.d/borg_sudoers</code>:</p>
<div class="highlight"><pre><span></span><code>demure <span class="nv">ALL</span><span class="o">=(</span>ALL<span class="o">)</span> NOPASSWD: /usr/local/sbin/borg_wrap.sh -q, /usr/local/sbin/borg_wrap.sh
</code></pre></div>
<p>Compared to my old <code>cryptshotr</code> it is very simple and concise.</p>
<p>Notable parts:</p>
<ul>
<li><code>borg_wrap.sh</code> is moved to <code>/usr/local/sbin</code> to prevent edits for user escalation.</li>
<li><code>/etc/sudoers.d</code> is used to prevent issues with bad edits breaking <code>sudo</code> completely, and on some OS's to survive <code>sudo</code> upgrades.</li>
<li>NOPASSWD is used so that it can run automated.</li>
</ul>
<h4 id="crontab">crontab</h4>
<p>I have <code>borg_wrap.sh</code> called every hour, and unlike my old <code>cryptshotr</code> I don't bother with fancy time checks as I don't need to balance/track multiple periodicities.</p>
<div class="highlight"><pre><span></span><code><span class="c1">## borgbackup wrapper</span>
<span class="m">0</span> */1 * * * sudo /usr/local/sbin/borg_wrap.sh -q
</code></pre></div>
<p><strong>Note</strong>: Aside from how often you run backups, periodicities are partially a function of how you prune.</p>
<h5 id="pruning">Pruning</h5>
<p>While I have not accumulated enough backups to use this yet, this is close to what I will be using:</p>
<div class="highlight"><pre><span></span><code>borg prune -v --list --dry-run -d<span class="o">=</span><span class="m">7</span> -w<span class="o">=</span><span class="m">4</span> -m<span class="o">=</span><span class="m">12</span> -y<span class="o">=</span>-1 --keep-within<span class="o">=</span>1d /path/to/repo
</code></pre></div>
<p>Which will keep:</p>
<ul>
<li>all archives <= 24hours old</li>
<li>one archive per day <= 7days old</li>
<li>one archive per week <= 4weeks old</li>
<li>one archive per month <= 12months old</li>
<li>unlimited yearly archives</li>
</ul>
<h4 id="conky_1">conky</h4>
<p>As a finishing touch, I updated my <code>back_log_backups.conkyrc</code> to use my <code>borg_wrap.sh</code> log, stored at <code>$HOME/.config/borg/log</code>.
This will show the last six entries, and if none six are a 'passed' it will show a seventh line at the top, showing the last 'passed' entry.
<img alt="borg conky" class="img-responsive" src="https://demu.red/pics/borg-conky.png"/></p>
<p><a href="https://notabug.org/demure/dotfiles/src/master/i3/conky/back_backups.conkyrc">back_log_backups.conkyr code</a></p>
<h3 id="borg_wrap.sh diagram_1">borg_wrap.sh Diagram</h3>
<p>So, as you can see, my setup is much more simplified now:
<img alt="borg_wrap.sh diagram" class="img-responsive" src="https://demu.red/pics/backups-borg_wrap.png"/></p>
<p><sup><strong>2017JUL09:</strong> Updated picture as I noticed an error.</sup></p>
<h3 id="afterword">Afterword</h3>
<p>Having done all the leg work for my laptop, it was easy to add another repo/authorized_key for my VPS and just tweak the script (and disable the wifi MAC address lookup).
So now I have both my laptop and my VPS backed up on my NAS.</p>
<h3 id="links">Links</h3>
<p><a href="https://demu.red/articles/008-backups_conky_display.md">my first post about backups</a><br/>
<a href="https://wiki.archlinux.org/index.php/Synchronization_and_backup_programs">Lot of Options</a><br/>
<a href="http://rsnapshot.org/">rsnapshot</a><br/>
<a href="https://notabug.org/demure/cryptshotr/src/master/cryptshotr">cryptshotr</a><br/>
<a href="https://github.com/pigmonkey/backups/blob/master/cryptshot.sh">cryptshot</a><br/>
<a href="https://twitter.com/thomasjwaldmann?lang=en">ThomasWaldmann</a><br/>
<a href="https://github.com/borgbackup/borg">BorgBackup</a><br/>
<a href="https://en.wikipedia.org/wiki/Rube_Goldberg_machine">Rube Goldberg-esk</a><br/>
<a href="https://github.com/jborg/attic">Attic</a><br/>
<a href="https://borgbackup.readthedocs.io/en/stable/faq.html#what-are-the-differences-between-attic-and-borg">What are the differences between Attic and Borg?</a><br/>
<a href="https://wiki.archlinux.org/index.php/autofs">AutoFS</a><br/>
<a href="https://github.com/Entware-ng/Entware-ng">Entware-ng</a><br/>
<a href="https://openwrt.org/">OpenWRT</a><br/>
<a href="https://github.com/borgbackup/borg/issues/690">issues</a><br/>
<a href="https://forum.qnap.com/viewtopic.php?t=115258#p596749">posted</a><br/>
<a href="https://www.passwordstore.org/">passwordstore</a><br/>
<a href="https://notabug.org/demure/scripts/src/master/borg_wrap.sh">borg_wrap.sh code</a><br/>
<a href="https://notabug.org/demure/dotfiles/src/master/i3/conky/back_backups.conkyrc">back_log_backups.conkyr code</a> </p>Setting up SMS in IRC, via BitlBee with purple-hangouts2016-12-26T01:19:00-05:002016-12-26T01:19:00-05:00demuretag:demu.red,2016-12-26:/blog/2016/12/setting-up-sms-in-irc-via-bitlbee-with-purple-hangouts/<p><strong>NOTE: It appears that this will cease to work as of 2021-01-27 according to a recent google email I received :'(</strong></p>
<h3 id="i've wanted this for so very long">I've wanted this for so very long</h3>
<p>Putting my texting into IRC, even if I don't send many messages, has long been a dream of mine.
Periodically I would look …</p><p><strong>NOTE: It appears that this will cease to work as of 2021-01-27 according to a recent google email I received :'(</strong></p>
<h3 id="i've wanted this for so very long">I've wanted this for so very long</h3>
<p>Putting my texting into IRC, even if I don't send many messages, has long been a dream of mine.
Periodically I would look into solutions, and be disappointed.
This week I picked up the torch again and came across two options.
The first option is pretty hacky, but I'm going to bring it up for those who can't currently do the second, and to show that the second best solution still left a lot to be desired.</p>
<h3 id="options">Options</h3>
<h4 id="gtalksms"><strike>GTalkSMS</strike></h4>
<p><a href="https://play.google.com/store/apps/details?id=com.googlecode.gtalksms&hl=en">GTalkSMS</a> seemed like a ray of light when I found it.
And it does work... But it is hacky...</p>
<h5 id="how does gtalksms work?">How does GTalkSMS work?</h5>
<p>Well, it is an android app that monitors for incoming SMS messages (among a handful of other features).
The hackyness comes into play with how the SMS messages get to IRC.</p>
<p>GTalkSMS requires <strong><em>two</em></strong> jabber accounts -- each account listing the other as a friend.</p>
<ul>
<li>You feed GTalkSMS the server/port/user/pass for the first account, and it will then connect to jabber as that account.</li>
<li>You teach it the name of the second account; this account will get notifications sent to it and be authorized to use 'commands'.</li>
</ul>
<p>So it pretty much acts as a relay, and forces your phone to use battery to be connected to jabber.</p>
<p><strong>Note:</strong> Getting to IRC at this point would involve connecting to <strong><em>just</em></strong> the second account in BitlBee. If You were connected to <em>both</em>, it would be spammy whenever you send a command to GTalkSMS... Which means you need a throwaway jabber account for this solution.</p>
<h4 id="so... what else can i find?_1">So... What else can I find?</h4>
<p>The above really didn't appeal to me, so I went back to researching.
After a while I came across <a href="https://bitbucket.org/EionRobb/purple-hangouts/">purple-hangouts</a>, and found that it did in fact handle SMS.</p>
<h4 id="purple-hangouts">purple-hangouts</h4>
<blockquote>
<p>Hangouts Plugin for libpurple<br/>
A replacement prpl for Hangouts in Pidgin/libpurple to support the proprietary protocol that Google uses for its Hangouts service. So far it supports all the fun things that aren't part of the XMPP interface, such as Group Chats, synchronised history between devices and SMS support via Google Voice. </p>
</blockquote>
<p>So, this can be used with <a href="https://www.bitlbee.org/main.php/news.r.html">BitlBee</a>, when it has been compiled to work with <a href="https://developer.pidgin.im/wiki/WhatIsLibpurple">libpurple</a>.
This was music to my ears.</p>
<h3 id="setup_1">Setup</h3>
<h4 id="installing bitlbee">Installing Bitlbee</h4>
<p>I'm going to fast forward through this step, as I already had a working BitlBee service running.
I did have to uninstall the <code>bitlbee</code> package, and replace it with <code>bitlbee-libpurpe</code>, on my Debian sid VPS.
This dropped right into place with the same config and non libpurple plugins (<a href="https://github.com/bitlbee/bitlbee-facebook">bitlbee-facebook</a> and <a href="https://github.com/bitlbee/bitlbee-steam">bitlbee-steam</a>).
When restarting BitlBee, to make sure the update worked, I ran into a snafu I've had with Debian's BitlBee install.
Namely, it seems like the <code>systemd</code> implementation of the daemon is borked.
To enact a restart of BitlBee I have to manually kill the PIDs, as <code>service bitlbee stop</code> does not work.
Yes, I know <code>systemctl</code> is the current command, but <code>service</code> is pretty much an alias.
After killing the PIDs, I then <code>service bitlbee stop</code> and <code>service bitlbee start</code>.
I suspect it is still using <code>initd</code> type files.</p>
<h4 id="installing purple-hangouts">Installing purple-hangouts</h4>
<p>On Debian, you will need the following dependencies: <code>libpurple-dev</code> <code>libjson-glib-dev</code> <code>libglib2.0-dev</code> <code>libprotobuf-c-dev</code> <code>protobuf-c-compiler</code> <code>mercurial</code> and <code>make</code>.
The <a href="https://bitbucket.org/EionRobb/purple-hangouts/overview">installation instructions</a> give a nice little code block for your copypasta pleasure, which I'll put below <sup>(using aptitude, as I prefer it)</sup>.</p>
<div class="highlight"><pre><span></span><code>sudo aptitude install libpurple-dev libjson-glib-dev libglib2.0-dev libprotobuf-c-dev protobuf-c-compiler mercurial make<span class="p">;</span>
hg clone https://bitbucket.org/EionRobb/purple-hangouts/ <span class="o">&&</span> <span class="nb">cd</span> purple-hangouts<span class="p">;</span>
make <span class="o">&&</span> sudo make install
</code></pre></div>
<p>Yes... this does force you to use <code>mercurial</code>... *<em>sigh</em>*<br/>
<strong>Note:</strong> EionRobb points out you can use <code>hg-git</code>, as well as downloading the source zip off of bitbucket.
I will have to give <code>hg-git</code> a try next time.</p>
<p>Anyway, installing was pretty easy, and I restarted BitlBee again to make sure it propagated.</p>
<h4 id="configuring google voice and google fi">Configuring Google Voice and Google Fi</h4>
<h5 id="google voice">Google Voice</h5>
<p>With a Google Voice set to use Hangouts, you are pretty much done.
Just make the Hangouts app your default SMS app.</p>
<h4 id="google fi_1">Google Fi</h4>
<p>In addition to setting the Hangouts app to be the default, you will need to adjust a Fi specific setting.
From Google's <a href="https://support.google.com/fi/answer/6188337?hl=en">Using Hangouts with Project Fi</a> page:</p>
<blockquote>
<ol>
<li>On your phone, open the Hangouts app </li>
<li>In the upper left corner, touch the Menu icon </li>
<li>Touch Settings </li>
<li>Touch the Google Account you use with Project Fi </li>
<li>Under “Project Fi calls and SMS,” check the box next to “Messages” and confirm </li>
</ol>
</blockquote>
<h4 id="added account to bitlbee">Added account to BitlBee</h4>
<p>At this point I removed my gtalk protocol account from BitlBee (<code>acc gtalk del</code>) and readded it as a hangouts protocol account, <code>acc add hangouts email@gmail.com</code>. </p>
<p>After connecting (<code>acc hangouts on</code>) you will get a privmsg from <code>purple_request_0</code>:</p>
<blockquote>
<p>2016-12-24 22:41:58 purple_request_0 Authorization Code<br/>
2016-12-24 22:41:58 purple_request_0 URL_HERE<br/>
2016-12-24 22:41:58 purple_request_0 Please follow the YouTube video to get the OAuth code </p>
</blockquote>
<p>I'll leave you to going through the video <sup>(link redacted, as it is a private youtube video)</sup>.
Once you have the oauth key, you paste it in and reply to <code>purple_request_0</code>.</p>
<p>And now your are connected!</p>
<h4 id="fixing buddy nicks">Fixing buddy nicks</h4>
<p>At this point, you will notice that your buddies' nicks are messed up.
To fix this you will want to do <code>acc hangouts set nick_format %full_name</code>, and then <code>acc hangouts off</code>, followed by <code>acc hangouts on</code>, to refresh the nicks.
Don't forget to use the <code>save</code> command to be on the safe side.</p>
<p><strong>Note:</strong> Alternatively, if you have pre-read these instructions, you could do this setting after adding the account and before connecting *<em>wink</em>*</p>
<h4 id="cell phone nicks">Cell Phone Nicks</h4>
<p>Unfortunately, at this point cell phones are not connected to the contact in purple-hangouts.
So if you have a friend named Alice who's number is 555-555-5555, you will have both <code>AliceAwesome</code> and <code>_155555555555</code>.
You can go through and <code>rename</code> the cell buddies to have a better nick with <code>rename _155555555555 AliceA_phone</code>.</p>
<h3 id="done!_1">Done!</h3>
<p>At this point you should be able to see sweet, sweet IRC side SMS messages!
I would like to thank Eion Robb and Mike 'Maiku' Ruprecht for making this beautiful tool.
I've been looking for something like this for <strong>years</strong>.</p>
<ul>
<li>☑ Life Goal: SMS via IRC </li>
<li>☐ Life Goal: Get all friends to use IRC</li>
</ul>
<h3 id="bonus round">Bonus Round</h3>
<p>My personal IRC setup involves <code>tmux</code> with <code>WeeChat</code> running inside it 24/7/365.
With <a href="https://weechat.org/">WeeChat</a>, I use <code>pushover.pl</code> <sup>(you can find it with the /script command)</sup> to send <a href="https://pushover.net/">pushover</a> notices to my phone on hilight/privmsg, when I am <code>/away</code>.
I then have my <code>/away</code> auto set by <code>screen_away.py</code> when I am detached from tmux, and not connected to my <a href="https://weechat.org/files/doc/stable/weechat_user.en.html#relay_plugin">weechat relay</a>.</p>
<p>So, how to I stop my phone from making a bing sound twice, once for the SMS and once for pushover?
Since 99% of renamed cells have a default prefix ('<strong>_1</strong>55555555555') <sup>(I'm assuming the '1', as I'm in the US)</sup>, and I use a standard convention when I set the nick ('name<strong>_phone</strong>'), this is doable using <code>plugins.var.perl.pushover.blacklist</code></p>
<div class="highlight"><pre><span></span><code><span class="x">/set plugins.var.perl.pushover.blacklist = "*_phone,_1*"</span>
</code></pre></div>
<p>And then <code>/save</code> for good measure.</p>
<h3 id="links">Links</h3>
<p><a href="https://play.google.com/store/apps/details?id=com.googlecode.gtalksms&hl=en">GTalkSMS</a><br/>
<a href="https://bitbucket.org/EionRobb/purple-hangouts/">purple-hangouts</a><br/>
<a href="https://www.bitlbee.org/main.php/news.r.html">BitlBee</a><br/>
<a href="https://developer.pidgin.im/wiki/WhatIsLibpurple">libpurple</a><br/>
<a href="https://github.com/bitlbee/bitlbee-facebook">bitlbee-facebook</a><br/>
<a href="https://github.com/bitlbee/bitlbee-steam">bitlbee-steam</a><br/>
<a href="https://bitbucket.org/EionRobb/purple-hangouts/overview">installation instructions</a><br/>
<a href="https://support.google.com/fi/answer/6188337?hl=en">Using Hangouts with Project Fi</a><br/>
<a href="https://weechat.org/">WeeChat</a><br/>
<a href="https://pushover.net/">pushover</a><br/>
<a href="https://weechat.org/files/doc/stable/weechat_user.en.html#relay_plugin">weechat relay</a> </p>Blocking IPv6 Ranges with iptables2016-09-15T21:56:00-04:002016-09-15T21:56:00-04:00demuretag:demu.red,2016-09-15:/blog/2016/09/blocking-ipv6-ranges-with-iptables/<h3 id="bots... bots as far as the eye can see...">Bots... Bots as Far as the Eye Can See...</h3>
<p>My fight with bots seems eternal... Likely because it will be.</p>
<h4 id="counter measures">Counter Measures</h4>
<p>I recently have started improving my <a href="https://notabug.org/demure/demu.red/src/master/content/extra/robots.txt">robots.txt</a>, and made a fail2ban jail for <a href="https://notabug.org/demure/dotfiles/src/master/system_wide/etc/fail2ban/filter.d/nginx-badbots.conf">sketchy scrapers/bots</a>.
<sup>(If I get to it, fail2ban will be a whole other …</sup></p><h3 id="bots... bots as far as the eye can see...">Bots... Bots as Far as the Eye Can See...</h3>
<p>My fight with bots seems eternal... Likely because it will be.</p>
<h4 id="counter measures">Counter Measures</h4>
<p>I recently have started improving my <a href="https://notabug.org/demure/demu.red/src/master/content/extra/robots.txt">robots.txt</a>, and made a fail2ban jail for <a href="https://notabug.org/demure/dotfiles/src/master/system_wide/etc/fail2ban/filter.d/nginx-badbots.conf">sketchy scrapers/bots</a>.
<sup>(If I get to it, fail2ban will be a whole other post.)</sup>
I've also started adding some of the more annoying ones to my iptables.</p>
<h4 id="sand...">Sand...</h4>
<p>After adding a particularly annoying scraper called <code>SeznamBot</code> to my <code>robots.txt</code>, it was still pinging my <code>robots.txt</code> file far too often for my taste.
So I added its thankfully listed ipv4 blocks to my iptables.
After doing that it made me more annoyed... It started scraping my site on IPv6, and was not honoring my <code>robots.txt</code> any more...</p>
<h3 id="running into a wall_1">Running Into a Wall</h3>
<p>When I decided to start blocking their IPv6 ranges, I ran into a wall.
In hindsight, it was probably a self imposed wall.
I'm going to blame this mental block on my Cisco IOS class from over the summer.</p>
<p>Anyway, <em>"How do I block an IPv6 range with ip6tables?"</em>...
Well, this question really turned up <strong><em>Nothing</em></strong>; nothing at all despite my best google-fu...
Which is why I'm doing this write up.</p>
<h3 id="simpler than you think">Simpler Than You Think</h3>
<p>After a day of searching I stumbled upon a IPv6 /CIDR table <sup>(Added at end of post)</sup>, and then it clicked in my head.
You block ranges the same as IPv4, you just have to deal with a <strong><em>lot</em></strong> more classless ranges. <em>A lot more.</em></p>
<p>It is at this point I'm going to rant about Cisco's IOS being stupid, and causing this brain fart for me...
For some incomprehensible reason to me, Cisco commands don't let you use /CIDR when dealing with IPv6.
<strong><scowls at cisco></strong></p>
<h4 id="solution">Solution</h4>
<p>And here are the three rules I wanted.</p>
<div class="highlight"><pre><span></span><code>ip6tables -A INPUT -j DROP -s 2a02:598:a::78:00/112 -m comment --comment <span class="s2">"SeznamBot"</span>
ip6tables -A INPUT -j DROP -s 2a02:598:a::79:00/112 -m comment --comment <span class="s2">"SeznamBot"</span>
ip6tables -A INPUT -j DROP -s 2a02:598:2::00/112 -m comment --comment <span class="s2">"SeznamBot"</span>
</code></pre></div>
<p>So I now have one bot blocked, ~∞ left to go!</p>
<h4 id="bonus: you can comment rules">Bonus: You Can Comment Rules</h4>
<p>As you might have just noticed in the above rules, you can add comments directly to your rules with a <code>-m comment --comment "Blah"</code>.
These comments will then show when you print your tables.</p>
<div class="highlight"><pre><span></span><code><span class="x">target prot opt source destination</span>
<span class="x">DROP all 2a02:598:a::78:0/112 anywhere /* SeznamBot */</span>
<span class="x">DROP all 2a02:598:a::79:0/112 anywhere /* SeznamBot */</span>
<span class="x">DROP all 2a02:598:2::/112 anywhere /* SeznamBot */</span>
</code></pre></div>
<h4 id="a fancy table: brought to you by markdown">A Fancy Table: Brought to You by Markdown</h4>
<blockquote>
<p>Table's Source <a href="https://www.mediawiki.org/wiki/Help:Range_blocks/IPv6">MediaWiki - Range blocks/IPv6</a></p>
</blockquote>
<table class="table table-striped table-hover">
<thead>
<tr>
<th align="left">CIDR</th>
<th align="left">Start of range</th>
<th align="left">End of range</th>
<th align="left">Total addresses</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">::/0</td>
<td align="left">::</td>
<td align="left">ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">340,282,366,920,938,463,463,374,607,431,768,211,456</td>
</tr>
<tr>
<td align="left">::/1</td>
<td align="left">::</td>
<td align="left">7fff:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">170,141,183,460,469,231,731,687,303,715,884,105,728</td>
</tr>
<tr>
<td align="left">::/2</td>
<td align="left">::</td>
<td align="left">3fff:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">85,070,591,730,234,615,865,843,651,857,942,052,864</td>
</tr>
<tr>
<td align="left">2000::/3</td>
<td align="left">2000::</td>
<td align="left">3fff:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">42,535,295,865,117,307,932,921,825,928,921,026,432</td>
</tr>
<tr>
<td align="left">2000::/4</td>
<td align="left">2000::</td>
<td align="left">2fff:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">21,267,647,932,558,653,966,460,912,964,485,513,216</td>
</tr>
<tr>
<td align="left">2000::/5</td>
<td align="left">2000::</td>
<td align="left">27ff:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">10,633,823,966,279,326,983,230,456,482,242,756,608</td>
</tr>
<tr>
<td align="left">2000::/6</td>
<td align="left">2000::</td>
<td align="left">23ff:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">5,316,911,983,139,663,491,615,228,241,121,378,304</td>
</tr>
<tr>
<td align="left">2000::/7</td>
<td align="left">2000::</td>
<td align="left">21ff:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">2,658,455,991,569,831,745,807,614,120,560,689,152</td>
</tr>
<tr>
<td align="left">2000::/8</td>
<td align="left">2000::</td>
<td align="left">20ff:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">1,329,227,995,784,915,872,903,807,060,280,344,576</td>
</tr>
<tr>
<td align="left">2000::/9</td>
<td align="left">2000::</td>
<td align="left">207f:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">664,613,997,892,457,936,451,903,530,140,172,288</td>
</tr>
<tr>
<td align="left">2000::/10</td>
<td align="left">2000::</td>
<td align="left">203f:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">332,306,998,946,228,968,225,951,765,070,086,144</td>
</tr>
<tr>
<td align="left">2000::/11</td>
<td align="left">2000::</td>
<td align="left">201f:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">166,153,499,473,114,484,112,975,882,535,043,072</td>
</tr>
<tr>
<td align="left">2000::/12</td>
<td align="left">2000::</td>
<td align="left">200f:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">83,076,749,736,557,242,056,487,941,267,521,536</td>
</tr>
<tr>
<td align="left">2000::/13</td>
<td align="left">2000::</td>
<td align="left">2007:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">41,538,374,868,278,621,028,243,970,633,760,768</td>
</tr>
<tr>
<td align="left">2000::/14</td>
<td align="left">2000::</td>
<td align="left">2003:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">20,769,187,434,139,310,514,121,985,316,880,384</td>
</tr>
<tr>
<td align="left">2000::/15</td>
<td align="left">2000::</td>
<td align="left">2001:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">10,384,593,717,069,655,257,060,992,658,440,192</td>
</tr>
<tr>
<td align="left">2001::/16</td>
<td align="left">2001::</td>
<td align="left">2001:ffff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">5,192,296,858,534,827,628,530,496,329,220,096</td>
</tr>
<tr>
<td align="left">2001::/17</td>
<td align="left">2001::</td>
<td align="left">2001:7fff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">2,596,148,429,267,413,814,265,248,164,610,048</td>
</tr>
<tr>
<td align="left">2001::/18</td>
<td align="left">2001::</td>
<td align="left">2001:3fff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">1,298,074,214,633,706,907,132,624,082,305,024</td>
</tr>
<tr>
<td align="left">2001::/19</td>
<td align="left">2001::</td>
<td align="left">2001:1fff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">649,037,107,316,853,453,566,312,041,152,512</td>
</tr>
<tr>
<td align="left">2001::/20</td>
<td align="left">2001::</td>
<td align="left">2001:0fff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">324,518,553,658,426,726,783,156,020,576,256</td>
</tr>
<tr>
<td align="left">2001::/21</td>
<td align="left">2001::</td>
<td align="left">2001:07ff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">162,259,276,829,213,363,391,578,010,288,128</td>
</tr>
<tr>
<td align="left">2001::/22</td>
<td align="left">2001::</td>
<td align="left">2001:03ff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">81,129,638,414,606,681,695,789,005,144,064</td>
</tr>
<tr>
<td align="left">2001::/23</td>
<td align="left">2001::</td>
<td align="left">2001:01ff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">40,564,819,207,303,340,847,894,502,572,032</td>
</tr>
<tr>
<td align="left">2001::/24</td>
<td align="left">2001::</td>
<td align="left">2001:00ff:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">20,282,409,603,651,670,423,947,251,286,016</td>
</tr>
<tr>
<td align="left">2001::/25</td>
<td align="left">2001::</td>
<td align="left">2001:007f:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">10,141,204,801,825,835,211,973,625,643,008</td>
</tr>
<tr>
<td align="left">2001::/26</td>
<td align="left">2001::</td>
<td align="left">2001:003f:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">5,070,602,400,912,917,605,986,812,821,504</td>
</tr>
<tr>
<td align="left">2001::/27</td>
<td align="left">2001::</td>
<td align="left">2001:001f:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">2,535,301,200,456,458,802,993,406,410,752</td>
</tr>
<tr>
<td align="left">2001::/28</td>
<td align="left">2001::</td>
<td align="left">2001:000f:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">1,267,650,600,228,229,401,496,703,205,376</td>
</tr>
<tr>
<td align="left">2001::/29</td>
<td align="left">2001::</td>
<td align="left">2001:0007:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">633,825,300,114,114,700,748,351,602,688</td>
</tr>
<tr>
<td align="left">2001::/30</td>
<td align="left">2001::</td>
<td align="left">2001:0003:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">316,912,650,057,057,350,374,175,801,344</td>
</tr>
<tr>
<td align="left">2001::/31</td>
<td align="left">2001::</td>
<td align="left">2001:0001:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">158,456,325,028,528,675,187,087,900,672</td>
</tr>
<tr>
<td align="left">2001:db8::/32</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:ffff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">79,228,162,514,264,337,593,543,950,336</td>
</tr>
<tr>
<td align="left">2001:db8::/33</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:7fff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">39,614,081,257,132,168,796,771,975,168</td>
</tr>
<tr>
<td align="left">2001:db8::/34</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:3fff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">19,807,040,628,566,084,398,385,987,584</td>
</tr>
<tr>
<td align="left">2001:db8::/35</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:1fff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">9,903,520,314,283,042,199,192,993,792</td>
</tr>
<tr>
<td align="left">2001:db8::/36</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0fff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">4,951,760,157,141,521,099,596,496,896</td>
</tr>
<tr>
<td align="left">2001:db8::/37</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:07ff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">2,475,880,078,570,760,549,798,248,448</td>
</tr>
<tr>
<td align="left">2001:db8::/38</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:03ff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">1,237,940,039,285,380,274,899,124,224</td>
</tr>
<tr>
<td align="left">2001:db8::/39</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:01ff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">618,970,019,642,690,137,449,562,112</td>
</tr>
<tr>
<td align="left">2001:db8::/40</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:00ff:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">309,485,009,821,345,068,724,781,056</td>
</tr>
<tr>
<td align="left">2001:db8::/41</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:007f:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">154,742,504,910,672,534,362,390,528</td>
</tr>
<tr>
<td align="left">2001:db8::/42</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:003f:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">77,371,252,455,336,267,181,195,264</td>
</tr>
<tr>
<td align="left">2001:db8::/43</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:001f:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">38,685,626,227,668,133,590,597,632</td>
</tr>
<tr>
<td align="left">2001:db8::/44</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:000f:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">19,342,813,113,834,066,795,298,816</td>
</tr>
<tr>
<td align="left">2001:db8::/45</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0007:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">9,671,406,556,917,033,397,649,408</td>
</tr>
<tr>
<td align="left">2001:db8::/46</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0003:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">4,835,703,278,458,516,698,824,704</td>
</tr>
<tr>
<td align="left">2001:db8::/47</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0001:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">2,417,851,639,229,258,349,412,352</td>
</tr>
<tr>
<td align="left">2001:db8::/48</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:ffff:ffff:ffff:ffff:ffff</td>
<td align="left">1,208,925,819,614,629,174,706,176</td>
</tr>
<tr>
<td align="left">2001:db8::/49</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:7fff:ffff:ffff:ffff:ffff</td>
<td align="left">604,462,909,807,314,587,353,088</td>
</tr>
<tr>
<td align="left">2001:db8::/50</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:3fff:ffff:ffff:ffff:ffff</td>
<td align="left">302,231,454,903,657,293,676,544</td>
</tr>
<tr>
<td align="left">2001:db8::/51</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:1fff:ffff:ffff:ffff:ffff</td>
<td align="left">151,115,727,451,828,646,838,272</td>
</tr>
<tr>
<td align="left">2001:db8::/52</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0fff:ffff:ffff:ffff:ffff</td>
<td align="left">75,557,863,725,914,323,419,136</td>
</tr>
<tr>
<td align="left">2001:db8::/53</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:07ff:ffff:ffff:ffff:ffff</td>
<td align="left">37,778,931,862,957,161,709,568</td>
</tr>
<tr>
<td align="left">2001:db8::/54</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:03ff:ffff:ffff:ffff:ffff</td>
<td align="left">18,889,465,931,478,580,854,784</td>
</tr>
<tr>
<td align="left">2001:db8::/55</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:01ff:ffff:ffff:ffff:ffff</td>
<td align="left">9,444,732,965,739,290,427,392</td>
</tr>
<tr>
<td align="left">2001:db8::/56</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:00ff:ffff:ffff:ffff:ffff</td>
<td align="left">4,722,366,482,869,645,213,696</td>
</tr>
<tr>
<td align="left">2001:db8::/57</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:007f:ffff:ffff:ffff:ffff</td>
<td align="left">2,361,183,241,434,822,606,848</td>
</tr>
<tr>
<td align="left">2001:db8::/58</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:003f:ffff:ffff:ffff:ffff</td>
<td align="left">1,180,591,620,717,411,303,424</td>
</tr>
<tr>
<td align="left">2001:db8::/59</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:001f:ffff:ffff:ffff:ffff</td>
<td align="left">590,295,810,358,705,651,712</td>
</tr>
<tr>
<td align="left">2001:db8::/60</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:000f:ffff:ffff:ffff:ffff</td>
<td align="left">295,147,905,179,352,825,856</td>
</tr>
<tr>
<td align="left">2001:db8::/61</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0007:ffff:ffff:ffff:ffff</td>
<td align="left">147,573,952,589,676,412,928</td>
</tr>
<tr>
<td align="left">2001:db8::/62</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0003:ffff:ffff:ffff:ffff</td>
<td align="left">73,786,976,294,838,206,464</td>
</tr>
<tr>
<td align="left">2001:db8::/63</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0001:ffff:ffff:ffff:ffff</td>
<td align="left">36,893,488,147,419,103,232</td>
</tr>
<tr>
<td align="left">2001:db8::/64</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:ffff:ffff:ffff:ffff</td>
<td align="left">18,446,744,073,709,551,616</td>
</tr>
<tr>
<td align="left">2001:db8::/65</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:7fff:ffff:ffff:ffff</td>
<td align="left">9,223,372,036,854,775,808</td>
</tr>
<tr>
<td align="left">2001:db8::/66</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:3fff:ffff:ffff:ffff</td>
<td align="left">4,611,686,018,427,387,904</td>
</tr>
<tr>
<td align="left">2001:db8::/67</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:1fff:ffff:ffff:ffff</td>
<td align="left">2,305,843,009,213,693,952</td>
</tr>
<tr>
<td align="left">2001:db8::/68</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0fff:ffff:ffff:ffff</td>
<td align="left">1,152,921,504,606,846,976</td>
</tr>
<tr>
<td align="left">2001:db8::/69</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:07ff:ffff:ffff:ffff</td>
<td align="left">576,460,752,303,423,488</td>
</tr>
<tr>
<td align="left">2001:db8::/70</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:03ff:ffff:ffff:ffff</td>
<td align="left">288,230,376,151,711,744</td>
</tr>
<tr>
<td align="left">2001:db8::/71</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:01ff:ffff:ffff:ffff</td>
<td align="left">144,115,188,075,855,872</td>
</tr>
<tr>
<td align="left">2001:db8::/72</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:00ff:ffff:ffff:ffff</td>
<td align="left">72,057,594,037,927,936</td>
</tr>
<tr>
<td align="left">2001:db8::/73</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:007f:ffff:ffff:ffff</td>
<td align="left">36,028,797,018,963,968</td>
</tr>
<tr>
<td align="left">2001:db8::/74</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:003f:ffff:ffff:ffff</td>
<td align="left">18,014,398,509,481,984</td>
</tr>
<tr>
<td align="left">2001:db8::/75</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:001f:ffff:ffff:ffff</td>
<td align="left">9,007,199,254,740,992</td>
</tr>
<tr>
<td align="left">2001:db8::/76</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:000f:ffff:ffff:ffff</td>
<td align="left">4,503,599,627,370,496</td>
</tr>
<tr>
<td align="left">2001:db8::/77</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0007:ffff:ffff:ffff</td>
<td align="left">2,251,799,813,685,248</td>
</tr>
<tr>
<td align="left">2001:db8::/78</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0003:ffff:ffff:ffff</td>
<td align="left">1,125,899,906,842,624</td>
</tr>
<tr>
<td align="left">2001:db8::/79</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0001:ffff:ffff:ffff</td>
<td align="left">562,949,953,421,312</td>
</tr>
<tr>
<td align="left">2001:db8::/80</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:ffff:ffff:ffff</td>
<td align="left">281,474,976,710,656</td>
</tr>
<tr>
<td align="left">2001:db8::/81</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:7fff:ffff:ffff</td>
<td align="left">140,737,488,355,328</td>
</tr>
<tr>
<td align="left">2001:db8::/82</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:3fff:ffff:ffff</td>
<td align="left">70,368,744,177,664</td>
</tr>
<tr>
<td align="left">2001:db8::/83</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:1fff:ffff:ffff</td>
<td align="left">35,184,372,088,832</td>
</tr>
<tr>
<td align="left">2001:db8::/84</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0fff:ffff:ffff</td>
<td align="left">17,592,186,044,416</td>
</tr>
<tr>
<td align="left">2001:db8::/85</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:07ff:ffff:ffff</td>
<td align="left">8,796,093,022,208</td>
</tr>
<tr>
<td align="left">2001:db8::/86</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:03ff:ffff:ffff</td>
<td align="left">4,398,046,511,104</td>
</tr>
<tr>
<td align="left">2001:db8::/87</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:01ff:ffff:ffff</td>
<td align="left">2,199,023,255,552</td>
</tr>
<tr>
<td align="left">2001:db8::/88</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:00ff:ffff:ffff</td>
<td align="left">1,099,511,627,776</td>
</tr>
<tr>
<td align="left">2001:db8::/89</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:007f:ffff:ffff</td>
<td align="left">549,755,813,888</td>
</tr>
<tr>
<td align="left">2001:db8::/90</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:003f:ffff:ffff</td>
<td align="left">274,877,906,944</td>
</tr>
<tr>
<td align="left">2001:db8::/91</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:001f:ffff:ffff</td>
<td align="left">137,438,953,472</td>
</tr>
<tr>
<td align="left">2001:db8::/92</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:000f:ffff:ffff</td>
<td align="left">68,719,476,736</td>
</tr>
<tr>
<td align="left">2001:db8::/93</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0007:ffff:ffff</td>
<td align="left">34,359,738,368</td>
</tr>
<tr>
<td align="left">2001:db8::/94</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0003:ffff:ffff</td>
<td align="left">17,179,869,184</td>
</tr>
<tr>
<td align="left">2001:db8::/95</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0001:ffff:ffff</td>
<td align="left">8,589,934,592</td>
</tr>
<tr>
<td align="left">2001:db8::/96</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:ffff:ffff</td>
<td align="left">4,294,967,296</td>
</tr>
<tr>
<td align="left">2001:db8::/97</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:7fff:ffff</td>
<td align="left">2,147,483,648</td>
</tr>
<tr>
<td align="left">2001:db8::/98</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:3fff:ffff</td>
<td align="left">1,073,741,824</td>
</tr>
<tr>
<td align="left">2001:db8::/99</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:1fff:ffff</td>
<td align="left">536,870,912</td>
</tr>
<tr>
<td align="left">2001:db8::/100</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0fff:ffff</td>
<td align="left">268,435,456</td>
</tr>
<tr>
<td align="left">2001:db8::/101</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:07ff:ffff</td>
<td align="left">134,217,728</td>
</tr>
<tr>
<td align="left">2001:db8::/102</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:03ff:ffff</td>
<td align="left">67,108,864</td>
</tr>
<tr>
<td align="left">2001:db8::/103</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:01ff:ffff</td>
<td align="left">33,554,432</td>
</tr>
<tr>
<td align="left">2001:db8::/104</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:00ff:ffff</td>
<td align="left">16,777,216</td>
</tr>
<tr>
<td align="left">2001:db8::/105</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:007f:ffff</td>
<td align="left">8,388,608</td>
</tr>
<tr>
<td align="left">2001:db8::/106</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:003f:ffff</td>
<td align="left">4,194,304</td>
</tr>
<tr>
<td align="left">2001:db8::/107</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:001f:ffff</td>
<td align="left">2,097,152</td>
</tr>
<tr>
<td align="left">2001:db8::/108</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:000f:ffff</td>
<td align="left">1,048,576</td>
</tr>
<tr>
<td align="left">2001:db8::/109</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0007:ffff</td>
<td align="left">524,288</td>
</tr>
<tr>
<td align="left">2001:db8::/110</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0003:ffff</td>
<td align="left">262,144</td>
</tr>
<tr>
<td align="left">2001:db8::/111</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0001:ffff</td>
<td align="left">131,072</td>
</tr>
<tr>
<td align="left">2001:db8::/112</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:ffff</td>
<td align="left">65,536</td>
</tr>
<tr>
<td align="left">2001:db8::/113</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:7fff</td>
<td align="left">32,768</td>
</tr>
<tr>
<td align="left">2001:db8::/114</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:3fff</td>
<td align="left">16,384</td>
</tr>
<tr>
<td align="left">2001:db8::/115</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:1fff</td>
<td align="left">8,192</td>
</tr>
<tr>
<td align="left">2001:db8::/116</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:0fff</td>
<td align="left">4,096</td>
</tr>
<tr>
<td align="left">2001:db8::/117</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:07ff</td>
<td align="left">2,048</td>
</tr>
<tr>
<td align="left">2001:db8::/118</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:03ff</td>
<td align="left">1,024</td>
</tr>
<tr>
<td align="left">2001:db8::/119</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:01ff</td>
<td align="left">512</td>
</tr>
<tr>
<td align="left">2001:db8::/120</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:00ff</td>
<td align="left">256</td>
</tr>
<tr>
<td align="left">2001:db8::/121</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:007f</td>
<td align="left">128</td>
</tr>
<tr>
<td align="left">2001:db8::/122</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:003f</td>
<td align="left">64</td>
</tr>
<tr>
<td align="left">2001:db8::/123</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:001f</td>
<td align="left">32</td>
</tr>
<tr>
<td align="left">2001:db8::/124</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:000f</td>
<td align="left">16</td>
</tr>
<tr>
<td align="left">2001:db8::/125</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:0007</td>
<td align="left">8</td>
</tr>
<tr>
<td align="left">2001:db8::/126</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:0003</td>
<td align="left">4</td>
</tr>
<tr>
<td align="left">2001:db8::/127</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8:0000:0000:0000:0000:0000:0001</td>
<td align="left">2</td>
</tr>
<tr>
<td align="left">2001:db8::/128</td>
<td align="left">2001:db8::</td>
<td align="left">2001:db8::</td>
<td align="left">1</td>
</tr>
</tbody>
</table>
<h3 id="links_1">Links</h3>
<p><a href="https://notabug.org/demure/demu.red/src/master/content/extra/robots.txt">robots.txt</a><br/>
<a href="https://notabug.org/demure/dotfiles/src/master/system_wide/etc/fail2ban/filter.d/nginx-badbots.conf">sketchy scrapers/bots</a><br/>
<a href="https://www.mediawiki.org/wiki/Help:Range_blocks/IPv6">MediaWiki - Range blocks/IPv6</a> </p>Dealing with Facebook Emotes in BitlBee with WeeChat2016-08-30T15:30:00-04:002016-08-30T15:30:00-04:00demuretag:demu.red,2016-08-30:/blog/2016/08/dealing-with-facebook-emotes-in-bitlbee-with-weechat/<h3 id="weechat, bitlbee, and the evil facebook">WeeChat, BitlBee, and the Evil Facebook</h3>
<h4 id="weechat">WeeChat</h4>
<p>I have used <a href="https://weechat.org/">WeeChat</a> for a number of years.
Previously I had been an Irssi user, but I kept being told of the wonders of WeeChat.
After trying it, I was convinced; I now frequently proselytize others, as Weechat is Irssi+++.
I think …</p><h3 id="weechat, bitlbee, and the evil facebook">WeeChat, BitlBee, and the Evil Facebook</h3>
<h4 id="weechat">WeeChat</h4>
<p>I have used <a href="https://weechat.org/">WeeChat</a> for a number of years.
Previously I had been an Irssi user, but I kept being told of the wonders of WeeChat.
After trying it, I was convinced; I now frequently proselytize others, as Weechat is Irssi+++.
I think I've filled out a few punch cards, and can redeem a couple of sandwiches by now. :)</p>
<p>A small selection of the awesome things WeeChat has to offer:</p>
<ul>
<li><a href="http://dev.weechat.org/post/2008/10/25/Smart-IRC-join-part-quit-message-filter">Smartfilters</a><ul>
<li>Can make join/part spam a thing of the past</li>
<li>You can toggle them off</li>
</ul>
</li>
<li>Edit settings from <em>inside</em> WeeChat<ul>
<li>Installing <code>/iset</code> makes it even better, as it explains settings</li>
</ul>
</li>
<li>Vertical and horizontal splits</li>
<li>Nick list is built in</li>
<li>The buffers script</li>
<li>The script installation UI</li>
<li>I could go on...</li>
</ul>
<h4 id="bitlbee">BitlBee</h4>
<p><a href="https://www.bitlbee.org/">BitlBee</a> is a fancy way to connect to IM protocols (as well as twitter/FB).
It is basically an IRCD that acts as a gateway.
I use it with my <a href="https://tmux.github.io/">tmux</a>+WeeChat setup, to remain online 24/7/365.
It works quite well.</p>
<h4 id="evil fb">Evil FB</h4>
<p>In addition to my various IM accounts, I have BitlBee do twitter and FB (notice how I'm not acknowledging FB as an IM protocol :p).
As much as it pains me to use FB, a few of my friends don't bother with real IMs anymore... TT__TT
And at least this way it's my server connecting to FB.</p>
<p>Why don't I consider it a <em>real</em> IM protocol?
Well, they started by using jabber, but have bastardized it over time.
Now they are at the point where I need to compile a special plugin for my BitlBee to connect.</p>
<h3 id="irksome expressive pictures_1">Irksome Expressive Pictures</h3>
<p>FB has had their in-chat pictures for a while.
Unfortunately, unless they are actual unicode, they end up being a link in BitlBee's chat window.
Here is how I see a <code>Thumbs Up</code> in a chat.</p>
<div class="highlight"><pre><span></span><code><span class="x">2016-08-27 22:32:46 Name_here https://fb-s-d-a.akamaihd.net/h-ak-xfa1/t39.1997-6/851557_369239266556155_759568595_n.png?_nc_ad=z-m</span>
</code></pre></div>
<p>Worse is that the links are not consistent.
They seem to both have duplicate locations and varying sizes...</p>
<div class="highlight"><pre><span></span><code><span class="x">2016-08-26 17:50:21 Name_here https://scontent.xx.fbcdn.net/t39.1997-6/851557_369239266556155_759568595_n.png?_nc_ad=z-m</span>
</code></pre></div>
<h3 id="a solution to my madness">A Solution to my Madness</h3>
<p>WeeChat has a fancy, and powerful, command call: <code>/trigger</code>.
Among the things <code>/trigger</code> can do is replace text.
Here is my current solution:</p>
<div class="highlight"><pre><span></span><code><span class="c1">## The first is probably the most desired</span>
/trigger addreplace fb_thumb modifier <span class="s2">"weechat_print"</span> <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ notify_private && </span><span class="si">${</span><span class="nv">tg_message</span><span class="si">}</span><span class="s2"> =~ 851557_369239266556155_759568595_n.png"</span> <span class="s2">"/(.*)/</span><span class="si">${</span><span class="nv">tg_prefix</span><span class="si">}</span><span class="s2">\t:thumbs: </span><span class="si">${</span><span class="nv">tg_message</span><span class="si">}</span><span class="s2">"</span> <span class="s2">""</span>
/trigger addreplace fb_arg_cat modifier <span class="s2">"weechat_print"</span> <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ notify_private && </span><span class="si">${</span><span class="nv">tg_message</span><span class="si">}</span><span class="s2"> =~ 10541001_1498876623686801_1597277063_n.png"</span> <span class="s2">"/(.*)/</span><span class="si">${</span><span class="nv">tg_prefix</span><span class="si">}</span><span class="s2">\t:arg_cat: </span><span class="si">${</span><span class="nv">tg_message</span><span class="si">}</span><span class="s2">"</span> <span class="s2">""</span>
/trigger addreplace fb_sleep_cat modifier <span class="s2">"weechat_print"</span> <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ notify_private && </span><span class="si">${</span><span class="nv">tg_message</span><span class="si">}</span><span class="s2"> =~ 10541016_1498876550353475_1185278857_n.png"</span> <span class="s2">"/(.*)/</span><span class="si">${</span><span class="nv">tg_prefix</span><span class="si">}</span><span class="s2">\t:sleepy_cat: </span><span class="si">${</span><span class="nv">tg_message</span><span class="si">}</span><span class="s2">"</span> <span class="s2">""</span>
/trigger addreplace fb_strong_bunny modifier <span class="s2">"weechat_print"</span> <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ notify_private && </span><span class="si">${</span><span class="nv">tg_message</span><span class="si">}</span><span class="s2"> =~ 11891354_1659201864325047_1474050348_n.png"</span> <span class="s2">"/(.*)/</span><span class="si">${</span><span class="nv">tg_prefix</span><span class="si">}</span><span class="s2">\t:strong_bunny: </span><span class="si">${</span><span class="nv">tg_message</span><span class="si">}</span><span class="s2">"</span> <span class="s2">""</span>
</code></pre></div>
<h4 id="an explanation">An Explanation</h4>
<ul>
<li>The <code>:desc: ${tg_message}</code> portion will display your added description, while still showing you the original message.<ul>
<li>I chose this setup, as I initially had issues where the trigger would replace my own messages if they matched.<ul>
<li>That was since corrected, upon my divining that I could add a <code>${tg_tags} =~ notify_private &&</code> to my <code>"${tg_message} =~ 11891354_1659201864325047_1474050348_n.png"</code>.</li>
</ul>
</li>
</ul>
</li>
<li>It is also worth noting that I stuck to using the <code>file_name.png</code> as my regex.<ul>
<li>I am told that the user can affect the size of a picture; I'm not going to bother to configure to account for size.</li>
<li>This means that it should catch all URL variations, but entropy is also slightly decreased.<ul>
<li>What I mean there is that if someone, for some unknown reason, actually says <code>851557_369239266556155_759568595_n.png</code> in a conversation you will trigger this...</li>
</ul>
</li>
</ul>
</li>
</ul>
<h4 id="the result">The Result</h4>
<div class="highlight"><pre><span></span><code><span class="x">2016-08-28 01:57:30 Name_here :thumbs: https://fb-s-d-a.akamaihd.net/h-ak-xfa1/t39.1997-6/851557_369239266556155_759568595_n.png?_nc_ad=z-m</span>
</code></pre></div>
<p>Alternatively, you could remove the last <code>${tg_message}</code> and only get the replacement text.</p>
<div class="highlight"><pre><span></span><code>/trigger addreplace fb_thumb modifier <span class="s2">"weechat_print"</span> <span class="s2">"</span><span class="si">${</span><span class="nv">tg_tags</span><span class="si">}</span><span class="s2"> =~ notify_private && </span><span class="si">${</span><span class="nv">tg_message</span><span class="si">}</span><span class="s2"> =~ 851557_369239266556155_759568595_n.png"</span> <span class="s2">"/(.*)/</span><span class="si">${</span><span class="nv">tg_prefix</span><span class="si">}</span><span class="s2">\t:thumbs:"</span> <span class="s2">""</span>
</code></pre></div>
<p>This would result in:</p>
<div class="highlight"><pre><span></span><code><span class="x">2016-08-28 01:57:30 Name_here :thumbs:</span>
</code></pre></div>
<h3 id="links_1">Links</h3>
<p><a href="https://weechat.org/">WeeChat</a><br/>
<a href="http://dev.weechat.org/post/2008/10/25/Smart-IRC-join-part-quit-message-filter">Smartfilters</a><br/>
<a href="https://www.bitlbee.org/">BitlBee</a><br/>
<a href="https://tmux.github.io/">tmux</a> </p>How to save space with dpkg2016-08-24T09:26:00-04:002016-08-24T09:26:00-04:00demuretag:demu.red,2016-08-24:/blog/2016/08/how-to-save-space-with-dpkg/<p><sub>This will be a quick post. While the subject is simple it seemed worth sharing... As it was new information to me.</sub></p>
<h3 id="ocd: obsessively cleaning your directories">OCD: Obsessively Cleaning your Directories</h3>
<p>Some of you out there work on minimal setups (ie: low disk space, embedded, etc).
Others of you likely get annoyed by wasted …</p><p><sub>This will be a quick post. While the subject is simple it seemed worth sharing... As it was new information to me.</sub></p>
<h3 id="ocd: obsessively cleaning your directories">OCD: Obsessively Cleaning your Directories</h3>
<p>Some of you out there work on minimal setups (ie: low disk space, embedded, etc).
Others of you likely get annoyed by wasted disk space.
I'm just going to assume you fall in both categories, as you read my blog and must be an awesome person. :D <sub>(lets see if my copy editors let me keep this informal tone)</sub></p>
<p>Personally, I get annoyed at useless large disk usage.
House cleaning my filesystem also has the benefit of speeding up <a href="https://demu.red/blog/2016/05/backups-options-and-effort/">My Backups</a>.</p>
<p>I had been staring at my <code>/usr/share/doc/texlive-doc/</code> for a while, when running <code>ncdu</code> to check on my disk usage.
It was just <em>there</em>, taking up a gigabyte or so.</p>
<div class="highlight"><pre><span></span><code>~ -> du -hcs /usr/share/doc/texlive-doc/
<span class="m">1</span>.1G /usr/share/doc/texlive-doc/
<span class="m">1</span>.1G total
</code></pre></div>
<p>I have yet to ever read the <code>TeX</code> documentation... And I'm not sure if I ever will have the motivation <sup>(ie: a fancy project)</sup> to do so.
After a number of months of staring at the disk usage, I finally decided to figure out how to free it up.</p>
<h3 id="your daily dose of the esoteric and arcane: dpkg">Your Daily Dose of the Esoteric and Arcane: dpkg</h3>
<p>While this may not be the most outlandish, this is a reoccurring theme with my computer knowledge; perhaps it will be one with my blog as well. <em>wink</em></p>
<p>In my research on the proper way to deal with this issue on debian sid, I found that <code>dpkg</code> has the ability to exclude and include paths from being installed.
Meet my new friends <code>path-exclude</code> and <code>path-include</code>.
The use of these is as simple as adding a config to <code>/etc/dpkg/dpkg.cfg.d/</code>. </p>
<p><strong>Note:</strong> The order of your lines matters, as the later a line the higher priority it has over the previous lines.</p>
<h3 id="the config file">The Config File</h3>
<p><code>/etc/dpkg/dpkg.cfg.d/excludes</code></p>
<div class="highlight"><pre><span></span><code><span class="c1">## Drop locales except english and japanese</span>
path-exclude<span class="o">=</span>/usr/share/locale/*
path-include<span class="o">=</span>/usr/share/locale/en/*
path-include<span class="o">=</span>/usr/share/locale/en_US/*
path-include<span class="o">=</span>/usr/share/locale/ja/*
path-include<span class="o">=</span>/usr/share/locale/locale.alias
<span class="c1">## Drop translated manual pages except english and japanese</span>
path-exclude<span class="o">=</span>/usr/share/man/*
path-include<span class="o">=</span>/usr/share/man/man<span class="o">[</span><span class="m">1</span>-9<span class="o">]</span>/*
path-include<span class="o">=</span>/usr/share/man/en*/*
path-include<span class="o">=</span>/usr/share/man/ja*/*
<span class="c1">## Drop annoyingly large texlive-doc</span>
path-exclude<span class="o">=</span>/usr/share/doc/texlive-doc/*
</code></pre></div>
<p><a href="https://notabug.org/demure/dotfiles/src/master/system_wide/etc/dpgk/dpkg.cfg.d/excludes">My Config File</a></p>
<p>This saves me over a gigabyte of space from unused man pages, locales, and of course texlive-doc.
Disk space saving isn't immediate, every time you install or upgrade a package the rules get evaluated and the excludes happen.
Here I allow English and Japanese to install, one of these days I might actually learn the latter. </p>
<p><strong>Note:</strong> If you need immediate relief, you have two options.</p>
<ol>
<li>Reinstall all packages.<ul>
<li>This is likely to have a few issues, at least on sid.</li>
</ul>
</li>
<li>Carefully <code>rm</code> directories and files.<ul>
<li>Do so at your own risk, reading my blog comes with <strong><em>no warranties!</em></strong></li>
</ul>
</li>
</ol>
<h3 id="links">Links</h3>
<p><a href="https://demu.red/blog/2016/05/backups-options-and-effort/">My Backups</a><br/>
<a href="https://notabug.org/demure/dotfiles/src/master/system_wide/etc/dpgk/dpkg.cfg.d/excludes">My Config File</a> </p>How to check if your GPG key is in cache PART 22016-08-16T21:40:00-04:002016-08-16T21:40:00-04:00demuretag:demu.red,2016-08-16:/blog/2016/08/how-to-check-if-your-gpg-key-is-in-cache-part-2/<p><strong>Followup <a href="https://demu.red/blog/2017/03/how-to-check-if-your-smartcards-gpg-key-is-in-cache-part-3/">Part3</a></strong> </p>
<h3 id="gpg cache checking: deja vu">GPG Cache Checking: deja vu</h3>
<p>Until this week I had been using a <em>clean</em> way of checking if my laptop GPG key was cached.<br/>
...<br/>
If this doesn't sound like a familiar story, see <a href="https://demu.red/blog/2016/06/how-to-check-if-your-gpg-key-is-in-cache/">Part1</a>.</p>
<h3 id="a broken solution">A Broken Solution</h3>
<p>This week debian sid moved to using <code>gpg2</code> as <code>gpg</code>.
After …</p><p><strong>Followup <a href="https://demu.red/blog/2017/03/how-to-check-if-your-smartcards-gpg-key-is-in-cache-part-3/">Part3</a></strong> </p>
<h3 id="gpg cache checking: deja vu">GPG Cache Checking: deja vu</h3>
<p>Until this week I had been using a <em>clean</em> way of checking if my laptop GPG key was cached.<br/>
...<br/>
If this doesn't sound like a familiar story, see <a href="https://demu.red/blog/2016/06/how-to-check-if-your-gpg-key-is-in-cache/">Part1</a>.</p>
<h3 id="a broken solution">A Broken Solution</h3>
<p>This week debian sid moved to using <code>gpg2</code> as <code>gpg</code>.
After the upgrade I found that my <a href="https://demu.red/blog/2016/05/backups-options-and-effort/">Backups</a> were no longer passing, via my <a href="https://demu.red/blog/2016/07/a-conky-display-for-my-backups/">Conky Script</a>.
However my awk was still working when run in the terminal... odd.</p>
<div class="highlight"><pre><span></span><code>gpg-connect-agent <span class="s1">'keyinfo --list'</span> /bye <span class="m">2</span>>/dev/null <span class="p">|</span> awk <span class="s1">'BEGIN{CACHED=0} /^S/ {match($0, /S\sKEYINFO\s\S+\s\S\s\S+\s\S+\s(\S)\s\S\s\S+\s\S+\s\S/, m); if(m[1]==1){CACHED=1}} END{print CACHED}'</span>
</code></pre></div>
<h4 id="why it broke">Why it Broke</h4>
<p>When I upgraded, I moved from:</p>
<ul>
<li><code>gnupg</code> 1.4.20-6 to 2.1.14-5 <sup>gnupg2 was separate</sup></li>
<li><code>gnupg2</code> 2.1.11-7 to 2.1.14-5</li>
<li><code>gnupg-agent</code> 2.1.11-7 to 2.1.14-5</li>
</ul>
<p>Post upgrade, <code>gpg-connect-agent 'keyinfo --list' /bye</code> still works normally.</p>
<div class="highlight"><pre><span></span><code>~ -> gpg-connect-agent <span class="s1">'keyinfo --list'</span> /bye
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED D - - - P - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED D - - <span class="m">1</span> P - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED T REDACTEDREDACTEDREDACTEDREDACTED OPENPGP.1 - - - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED T REDACTEDREDACTEDREDACTEDREDACTED OPENPGP.3 - - - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED T REDACTEDREDACTEDREDACTEDREDACTED OPENPGP.2 - - - - -
OK
</code></pre></div>
<p>But <code>sudo gpg-connect-agent 'keyinfo --list' /bye</code> does not.</p>
<div class="highlight"><pre><span></span><code>~ -> gpg-connect-agent <span class="s1">'keyinfo --list'</span> /bye
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED D - - - P - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED D - - - P - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED T REDACTEDREDACTEDREDACTEDREDACTED OPENPGP.1 - - - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED T REDACTEDREDACTEDREDACTEDREDACTED OPENPGP.3 - - - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED T REDACTEDREDACTEDREDACTEDREDACTED OPENPGP.2 - - - - -
OK
</code></pre></div>
<p>After asking around on <code>#gnupg</code>, it looks like socket location was changed in version 2.1.13. <sup>(thank you K_F, for pointing me in the right direction)</sup>
Instead of using <code>$HOME/.gnupg/S.gpg-agent</code>, <code>gpg-agent</code> now uses <code>/run/user/${UID}/gnupg/S.gpg-agent</code>
Running <code>sudo gpg-connect-agent -S /run/user/${UID}/gnupg/S.gpg-agent 'keyinfo --list' /bye</code> produces the correct output again.</p>
<h3 id="a clean solution... revisited_1">A Clean Solution... revisited</h3>
<p>When <code>sudo</code> privileges are needed <sup>(otherwise things still work)</sup>, you can use:</p>
<div class="highlight"><pre><span></span><code>gpg-connect-agent -S /run/user/<span class="si">${</span><span class="nv">UID</span><span class="si">}</span>/gnupg/S.gpg-agent <span class="s1">'keyinfo --list'</span> /bye <span class="m">2</span>>/dev/null <span class="p">|</span> awk <span class="s1">'BEGIN{CACHED=0} /^S/ {match($0, /S\sKEYINFO\s\S+\s\S\s\S+\s\S+\s(\S)\s\S\s\S+\s\S+\s\S/, m); if(m[1]==1){CACHED=1}} END{print CACHED}'</span>
</code></pre></div>
<p>This is assuming you aren't <code>sudo -s</code>, where <code>$SUDO_UID</code> would then be needed.</p>
<p><strong>EDIT 19FEB2017:</strong> In hind sight, there is a better awk, which now doesn't have to be gawk:</p>
<div class="highlight"><pre><span></span><code>gpg-connect-agent <span class="s1">'keyinfo --list'</span> /bye <span class="m">2</span>>/dev/null <span class="p">|</span> awk <span class="s1">'BEGIN{CACHED=0} /^S/ {if($7==1){CACHED=1}} END{if($0!=""){print CACHED} else {print "none"}}'</span>
gpg-connect-agent -S /run/user/<span class="si">${</span><span class="nv">UID</span><span class="si">}</span>/gnupg/S.gpg-agent <span class="s1">'keyinfo --list'</span> /bye <span class="m">2</span>>/dev/null <span class="p">|</span> awk <span class="s1">'BEGIN{CACHED=0} /^S/ {if($7==1){CACHED=1}} END{print CACHED}'</span>
</code></pre></div>
<h3 id="afterwords">Afterwords</h3>
<p>While this does work for the cache check of my backups <sup>(with added code to account for <code>$UID</code> vs <code>$SUDU_UID</code>, as manually running and <code>cryptshotr</code> and running via <code>cryptshotr-cron</code> have different needs...), it does not handle <code>sudo gpg...</code> working...<br/>
<strong>sigh</strong><br/>
So I am currently still using a <em>hack</em>, which is to link <code>$HOME/.gnupg/S.gpg-agent</code> to it's current location. After this, Things-Just-Work™.<br/>
For now. </sup></p>
<div class="highlight"><pre><span></span><code>ln -s /run/user/<span class="si">${</span><span class="nv">UID</span><span class="si">}</span>/gnupg/S.gpg-agent <span class="nv">$HOME</span>/.gnupg/S.gpg-agent
</code></pre></div>
<h3 id="links">Links</h3>
<p><a href="https://demu.red/blog/2017/03/how-to-check-if-your-smartcards-gpg-key-is-in-cache-part-3/">Part3</a><br/>
<a href="https://demu.red/blog/2016/06/how-to-check-if-your-gpg-key-is-in-cache/">Part1</a><br/>
<a href="https://demu.red/blog/2016/05/backups-options-and-effort/">Backups</a><br/>
<a href="https://demu.red/blog/2016/07/a-conky-display-for-my-backups/">Conky script</a> </p>A conky display for my backups2016-07-04T21:19:00-04:002016-07-04T21:19:00-04:00demuretag:demu.red,2016-07-04:/blog/2016/07/a-conky-display-for-my-backups/<h3 id="status checks">Status Checks</h3>
<p>While working on and testing my <a href="https://demu.red/articles/006-backups_options_and_effort.md">Backup Solution</a>, I made use of a simple conky script that displayed my <code>~/.conky/cryptshotr-corn/log</code> on top of my desktop.
This was a fairly simple <a href="https://github.com/brndnmtthws/conky">conky</a>, which I cribbed from my syslog/auth conky script.
Due to the number of expected …</p><h3 id="status checks">Status Checks</h3>
<p>While working on and testing my <a href="https://demu.red/articles/006-backups_options_and_effort.md">Backup Solution</a>, I made use of a simple conky script that displayed my <code>~/.conky/cryptshotr-corn/log</code> on top of my desktop.
This was a fairly simple <a href="https://github.com/brndnmtthws/conky">conky</a>, which I cribbed from my syslog/auth conky script.
Due to the number of expected fails, like using my laptop while not at home, I taught <a href="https://notabug.org/demure/cryptshotr">cryptshotr</a> to clear old 'PERIODICITY failed' lines on pass.
This worked well enough.</p>
<div class="highlight"><pre><span></span><code><span class="x">2016-01-01 19:00:00 yearly passed 1451606400 ## Mock entry, hasn't been a year yet ^_-</span>
<span class="x">2016-06-05 21:34:06 monthly passed 1465176846</span>
<span class="x">2016-06-26 18:56:04 weekly passed 1466981764</span>
<span class="x">2016-07-01 10:25:27 daily passed 1467383127</span>
<span class="x">2016-07-01 11:20:19 hourly passed 1467386419</span>
<span class="x">2016-07-01 14:00:01 hourly FAILED 1467396001</span>
<span class="x">2016-07-01 14:30:01 hourly FAILED 1467397801</span>
<span class="x">2016-07-01 15:00:01 hourly FAILED 1467399601</span>
<span class="x">2016-07-01 15:30:01 hourly FAILED 1467401401</span>
<span class="x">2016-07-01 16:03:48 hourly FAILED 1467403428</span>
<span class="x">2016-07-01 18:00:01 hourly FAILED 1467410401</span>
<span class="x">2016-07-01 18:30:01 hourly FAILED 1467412201</span>
</code></pre></div>
<h3 id="extended absence">Extended Absence</h3>
<p>A month or so after finishing cyrptshotr I was house sitting and away from my place from about two weeks.
This generated a <em>lot</em> of expected fail messages...
Every half hour PERIODICITIES that were due would attempt to run, and as time went on the number due increased; this was further compounded by being the end of the month (hourly, daily, weekly, monthly).
As expected, that got a little bit annoying.
Now yes, I could have taken the backup drive with me and directly connected it, and I may do this in the future.
But that doesn't mean that the display can't be better.</p>
<h3 id="what could be better">What Could Be Better</h3>
<p>Instead of displaying everything why not:</p>
<ul>
<li>grab the last occurrence of PERIODICITY</li>
<li>see if it passed</li>
<li>display when it passed <ul>
<li>or find the last time PERIODICITY passed</li>
</ul>
</li>
<li>use a simple icon to show pass/fail</li>
</ul>
<h4 id="awk a beautiful beast">Awk <sup>a beautiful beast</sup></h4>
<div class="highlight"><pre><span></span><code>awk <span class="s1">'/PERIODICITY/ {LL=$0;if($4=="passed"){LAST=$1 " " $2}} END {match(LL, /(.*)\s(.*)\t(\w+)\t(\w+)\t([0-9]+)/, m); if(m[4]=="passed"){printf "☑ passed %s %s\n", m[1], m[2]} else {printf "⚠ last passed %s\n", LAST}}'</span> <span class="nv">$HOME</span>/.config/cryptshotr-cron/log
</code></pre></div>
<h3 id="end result_1">End Result</h3>
<p>My <a href="https://notabug.org/demure/dotfiles/src/master/i3/conky/back_backups.conkyrc">back_log_backups.conkyrc</a>:<br/>
<img alt="cryptshotr conky" class="img-responsive" src="https://demu.red/pics/cryptshotr-conky.png"/></p>
<div class="highlight"><pre><span></span><code><span class="c1">## Window settings</span>
own_window yes
own_window_type override
own_window_hints undecorated,below,sticky,skip_taskbar,skip_pager
double_buffer yes
<span class="c1">## Compton doesn't like conky transparent, so this is the fix</span>
<span class="c1">## http://conky.pitstop.free.fr/wiki/index.php5?title=Transparency_in_conky_(en)</span>
own_window_transparent no <span class="c1">#yes</span>
own_window_argb_visual yes
own_window_argb_value <span class="m">0</span>
<span class="c1">## Window Size</span>
minimum_width <span class="m">1900</span>
maximum_width <span class="m">1920</span>
<span class="c1"># fiddle with window</span>
use_spacer none
use_xft yes
<span class="c1">## Update interval in seconds</span>
update_interval <span class="m">60</span>.0
<span class="c1">## Minimum size of text area</span>
<span class="c1"># minimum_size 250 5</span>
<span class="c1">## Draw shades?</span>
draw_shades no
<span class="c1">## Text stuff</span>
draw_outline no <span class="c1"># amplifies text if yes</span>
draw_borders no
font PragmataPro:size<span class="o">=</span><span class="m">7</span>
uppercase no
<span class="c1">## Default colors and also border colors, grey90 == #e5e5e5</span>
default_color grey
<span class="c1">## Text alignment, other possible values are commented</span>
<span class="c1">#alignment top_left</span>
alignment top_right
<span class="c1">#alignment bottom_left</span>
<span class="c1">#alignment bottom_right</span>
<span class="c1">## Gap between borders of screen and text</span>
<span class="c1">## The default size is a bit large</span>
gap_x <span class="m">10</span>
gap_y <span class="m">10</span>
<span class="c1">## stuff after ‘TEXT’ will be formatted on screen</span>
<span class="c1">## This summarizes the backup pass/fail state.</span>
<span class="c1">## Written to display 'fail' if periodicity not in log, though will look odd.</span>
<span class="c1">## For some reason 'monthly' can't be regexed... though it worked before fixing fail state.</span>
<span class="c1">## As such, 'month' is intentional.</span>
TEXT
<span class="nv">$color</span>
<span class="si">${</span><span class="nv">color</span><span class="p"> grey90</span><span class="si">}</span>Backups <span class="si">${</span><span class="nv">hr</span><span class="p"> 2</span><span class="si">}</span><span class="nv">$color</span>
Yearly <span class="si">${</span><span class="nv">alignr</span><span class="si">}${</span><span class="nv">exec</span><span class="p"> awk </span><span class="s1">'/yearly/ {LL=$0;if($4=="passed"){LAST=$1 " " $2}} END {match(LL, /(.*)\s(.*)\t(\w+)\t(\w+)\t([0-9]+)/, m); if(m[4]=="passed"){printf "☑ passed %s %s\n", m[1], m[2]} else {printf "⚠ last passed %s\n", LAST}}'</span><span class="p"> </span><span class="nv">$HOME</span><span class="p">/.config/cryptshotr-cron/log</span><span class="si">}</span>
Monthly <span class="si">${</span><span class="nv">alignr</span><span class="si">}${</span><span class="nv">exec</span><span class="p"> awk </span><span class="s1">'/month/ {LL=$0;if($4=="passed"){LAST=$1 " " $2}} END {match(LL, /(.*)\s(.*)\t(\w+)\t(\w+)\t([0-9]+)/, m); if(m[4]=="passed"){printf "☑ passed %s %s\n", m[1], m[2]} else {printf "⚠ last passed %s\n", LAST}}'</span><span class="p"> </span><span class="nv">$HOME</span><span class="p">/.config/cryptshotr-cron/log</span><span class="si">}</span>
Weekly <span class="si">${</span><span class="nv">alignr</span><span class="si">}${</span><span class="nv">exec</span><span class="p"> awk </span><span class="s1">'/weekly/ {LL=$0;if($4=="passed"){LAST=$1 " " $2}} END {match(LL, /(.*)\s(.*)\t(\w+)\t(\w+)\t([0-9]+)/, m); if(m[4]=="passed"){printf "☑ passed %s %s\n", m[1], m[2]} else {printf "⚠ last passed %s\n", LAST}}'</span><span class="p"> </span><span class="nv">$HOME</span><span class="p">/.config/cryptshotr-cron/log</span><span class="si">}</span>
Daily <span class="si">${</span><span class="nv">alignr</span><span class="si">}${</span><span class="nv">exec</span><span class="p"> awk </span><span class="s1">'/daily/ {LL=$0;if($4=="passed"){LAST=$1 " " $2}} END {match(LL, /(.*)\s(.*)\t(\w+)\t(\w+)\t([0-9]+)/, m); if(m[4]=="passed"){printf "☑ passed %s %s\n", m[1], m[2]} else {printf "⚠ last passed %s\n", LAST}}'</span><span class="p"> </span><span class="nv">$HOME</span><span class="p">/.config/cryptshotr-cron/log</span><span class="si">}</span>
Hourly <span class="si">${</span><span class="nv">alignr</span><span class="si">}${</span><span class="nv">exec</span><span class="p"> awk </span><span class="s1">'/hourly/ {LL=$0;if($4=="passed"){LAST=$1 " " $2}} END {match(LL, /(.*)\s(.*)\t(\w+)\t(\w+)\t([0-9]+)/, m); if(m[4]=="passed"){printf "☑ passed %s %s\n", m[1], m[2]} else {printf "⚠ last passed %s\n", LAST}}'</span><span class="p"> </span><span class="nv">$HOME</span><span class="p">/.config/cryptshotr-cron/log</span><span class="si">}</span>
</code></pre></div>
<h3 id="links">Links</h3>
<p><a href="https://demu.red/articles/006-backups_options_and_effort.md">Backup Solution</a><br/>
<a href="https://github.com/brndnmtthws/conky">conky</a><br/>
<a href="https://notabug.org/demure/cryptshotr">cryptshotr</a><br/>
<a href="https://notabug.org/demure/dotfiles/src/master/i3/conky/back_backups.conkyrc">back_log_backups.conkyrc</a> </p>How to check if your GPG key is in cache2016-06-26T02:00:00-04:002016-06-26T02:00:00-04:00demuretag:demu.red,2016-06-26:/blog/2016/06/how-to-check-if-your-gpg-key-is-in-cache/<p><strong>Followup <a href="https://demu.red/blog/2016/08/how-to-check-if-your-gpg-key-is-in-cache-part-2/">Part2</a></strong><br/>
<strong>Followup <a href="https://demu.red/blog/2017/03/how-to-check-if-your-smartcards-gpg-key-is-in-cache-part-3/">Part3</a></strong> </p>
<h3 id="gpg cache checking">GPG Cache Checking</h3>
<p>Until this week I had been using a <em>hacky</em> way of checking if my laptop GPG key was cached.
I utilize this for my <a href="https://notabug.org/demure/dotfiles/src/master/i3/lemonbar">WM's bar</a> so I can tell if the key is open (or if I need to refresh it for …</p><p><strong>Followup <a href="https://demu.red/blog/2016/08/how-to-check-if-your-gpg-key-is-in-cache-part-2/">Part2</a></strong><br/>
<strong>Followup <a href="https://demu.red/blog/2017/03/how-to-check-if-your-smartcards-gpg-key-is-in-cache-part-3/">Part3</a></strong> </p>
<h3 id="gpg cache checking">GPG Cache Checking</h3>
<p>Until this week I had been using a <em>hacky</em> way of checking if my laptop GPG key was cached.
I utilize this for my <a href="https://notabug.org/demure/dotfiles/src/master/i3/lemonbar">WM's bar</a> so I can tell if the key is open (or if I need to refresh it for mutt/backups), as well as letting scripts cleanly fail if it is not cached (like <a href="https://demu.red/blog/2016/05/backups-options-and-effort/">My Backups</a>).
I call it a hack, as it <em>technically</em> worked, but I was using an implied reaction... Which broke this week.
Well, it is more that I corrected my GPG set up by clearing my yubikey neo, and reconfiguring it to work again, at which point the reaction was no longer as desired.</p>
<h4 id="digression: yubikey neo gpg reset">Digression: Yubikey Neo GPG Reset</h4>
<p>Resetting the GPG Smartcard side of the Yubikey Neo isn't immediately obvious.
What do I mean?
Well, in <code>gpg2 --card-edit</code> the <code>factory-reset</code> option is not supported.</p>
<p>So, the solution was to create a file containing the following:</p>
<div class="highlight"><pre><span></span><code><span class="x">/hex</span>
<span class="x">scd serialno</span>
<span class="x">scd apdu 00 20 00 81 08 40 40 40 40 40 40 40 40</span>
<span class="x">scd apdu 00 20 00 81 08 40 40 40 40 40 40 40 40</span>
<span class="x">scd apdu 00 20 00 81 08 40 40 40 40 40 40 40 40</span>
<span class="x">scd apdu 00 20 00 81 08 40 40 40 40 40 40 40 40</span>
<span class="x">scd apdu 00 20 00 83 08 40 40 40 40 40 40 40 40</span>
<span class="x">scd apdu 00 20 00 83 08 40 40 40 40 40 40 40 40</span>
<span class="x">scd apdu 00 20 00 83 08 40 40 40 40 40 40 40 40</span>
<span class="x">scd apdu 00 20 00 83 08 40 40 40 40 40 40 40 40</span>
<span class="x">scd apdu 00 e6 00 00</span>
<span class="x">scd apdu 00 44 00 00</span>
<span class="x">/echo Card has been successfully reset.</span>
</code></pre></div>
<p>And then run <code>gpg-connect-agent -f FILE_NAME</code>.
This wiped the smartcard clean, and allowed me to start fresh.</p>
<h3 id="broken hack_1">Broken Hack</h3>
<p>My previous solution, <code>echo "1234" | gpg2 --no-tty --quiet --batch --local-user KEY_HERE -as - >/dev/null 2>&1 && echo "1" || echo "0"</code>, was based on a snippet of code which was intended as a quick way to make sure you knew the correct passphrase.
I had noticed that if the key was already cached you would get an automatic pass, and was using this as a quick and dirty test.
Unfortunately (or perhaps fortunately, as it spurred me to find a clean way and write this post) this hack no longer worked after I reconfigured my Yubikey and left stubfiles on my laptop.</p>
<h4 id="my keys">My Keys</h4>
<p>I have a master key (not kept on laptop) and an encryption key made while creating the master key (which I don't use); my Signature, Encryption, and Authentication subkeys I purposely made; and my laptop Encryption subkey.
The <em>"Laptop Encryption key"</em> is a key I use for <code>pass</code> and <a href="https://demu.red/blog/2016/05/backups-options-and-effort/">My Backups</a>.
I allow this to stay on the laptop, but segregate its use to things I need to work automagically.
The first three subkeys live on my yubikey, and it must be inserted to use them.</p>
<h5 id="digression: master and subkeys">Digression: Master and Subkeys</h5>
<p>If you use GPG, you should use the Master and Subkey set up.
There are a number of benefits.
Two of the big ones:</p>
<ul>
<li>You can keep the master key offline<ul>
<li>This is used for new key creation, and web-of-trust signing</li>
</ul>
</li>
<li>You can revoke subkeys, while maintaning your master key's web-of-trust<ul>
<li>The alternative is to start from ground zero</li>
</ul>
</li>
</ul>
<p>There is a very long conversation to be had here.
Instead of going into it, I'm going to link you to a good post on the topic, <em><a href="https://alexcabal.com/creating-the-perfect-gpg-keypair/">Creating the Perfect GPG Keypair</a></em>.</p>
<h4 id="why it broke_1">Why it Broke</h4>
<p>The old hack fails now, as my Signature subkey no longer lives on the laptop.
I attempted to tweak the code to remove the need for signing, but couldn't get past it.</p>
<h3 id="a clean solution_1">A Clean Solution</h3>
<p>I had a feeling that GPG itself would need some kind of way of knowing a key was cached, so there had to be an interface somewhere.
After a few hours of exercising my Google Fu, interrupted by a social gathering, I noticed a few uses of <code>gpg-connect-agent</code> that made it looked promising.
The man page doesn't really cover the command adequately in my opinion.
It also fails to mention the normal interactive commands...
I found this out after <code>echo "help" | gpg-connect-agent</code> spit out a number of [^/]COMMANDS not listed in the man page.
At this point I guessed that it might actually have an interactive shell, and found that it did.
After reading through all the <code>help COMMAND</code> outputs, I saw a few that looked promising and used some more Google Fu.</p>
<div class="highlight"><pre><span></span><code>~ -> gpg-connect-agent <span class="s1">'help keyinfo'</span> /bye
<span class="c1"># KEYINFO [--[ssh-]list] [--data] [--ssh-fpr] [--with-ssh] <keygrip></span>
<span class="c1"># </span>
<span class="c1"># Return information about the key specified by the KEYGRIP. If the</span>
<span class="c1"># key is not available GPG_ERR_NOT_FOUND is returned. If the option</span>
<span class="c1"># --list is given the keygrip is ignored and information about all</span>
<span class="c1"># available keys are returned. If --ssh-list is given information</span>
<span class="c1"># about all keys listed in the sshcontrol are returned. With --with-ssh</span>
<span class="c1"># information from sshcontrol is always added to the info. Unless --data</span>
<span class="c1"># is given, the information is returned as a status line using the format:</span>
<span class="c1"># </span>
<span class="c1"># KEYINFO <keygrip> <type> <serialno> <idstr> <cached> <protection> <fpr></span>
<span class="c1"># </span>
<span class="c1"># KEYGRIP is the keygrip.</span>
<span class="c1"># </span>
<span class="c1"># TYPE is describes the type of the key:</span>
<span class="c1"># 'D' - Regular key stored on disk,</span>
<span class="c1"># 'T' - Key is stored on a smartcard (token),</span>
<span class="c1"># 'X' - Unknown type,</span>
<span class="c1"># '-' - Key is missing.</span>
<span class="c1"># </span>
<span class="c1"># SERIALNO is an ASCII string with the serial number of the</span>
<span class="c1"># smartcard. If the serial number is not known a single</span>
<span class="c1"># dash '-' is used instead.</span>
<span class="c1"># </span>
<span class="c1"># IDSTR is the IDSTR used to distinguish keys on a smartcard. If it</span>
<span class="c1"># is not known a dash is used instead.</span>
<span class="c1"># </span>
<span class="c1"># CACHED is 1 if the passphrase for the key was found in the key cache.</span>
<span class="c1"># If not, a '-' is used instead.</span>
<span class="c1"># </span>
<span class="c1"># PROTECTION describes the key protection type:</span>
<span class="c1"># 'P' - The key is protected with a passphrase,</span>
<span class="c1"># 'C' - The key is not protected,</span>
<span class="c1"># '-' - Unknown protection.</span>
<span class="c1"># </span>
<span class="c1"># FPR returns the formatted ssh-style fingerprint of the key. It is only</span>
<span class="c1"># printed if the option --ssh-fpr has been used. It defaults to '-'.</span>
<span class="c1"># </span>
<span class="c1"># TTL is the TTL in seconds for that key or '-' if n/a.</span>
<span class="c1"># </span>
<span class="c1"># FLAGS is a word consisting of one-letter flags:</span>
<span class="c1"># 'D' - The key has been disabled,</span>
<span class="c1"># 'S' - The key is listed in sshcontrol (requires --with-ssh),</span>
<span class="c1"># 'c' - Use of the key needs to be confirmed,</span>
<span class="c1"># '-' - No flags given.</span>
<span class="c1"># </span>
<span class="c1"># More information may be added in the future.</span>
OK
</code></pre></div>
<p>This is where I found that <code>keyinfo --list</code>, or non interactively <code>gpg-connect-agent 'keyinfo --list' /bye</code> will spit out some tasty info.</p>
<div class="highlight"><pre><span></span><code>~ -> gpg-connect-agent <span class="s1">'keyinfo --list'</span> /bye
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED D - - - P - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED D - - <span class="m">1</span> P - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED T REDACTEDREDACTEDREDACTEDREDACTED OPENPGP.1 - - - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED T REDACTEDREDACTEDREDACTEDREDACTED OPENPGP.3 - - - - -
S KEYINFO REDACTEDREDACTEDREDACTEDREDACTEDREDACTED T REDACTEDREDACTEDREDACTEDREDACTED OPENPGP.2 - - - - -
OK
</code></pre></div>
<p>Notice that singular '1'? </p>
<p><strong>YES!</strong></p>
<p>Now time to Regex this beauty!
And some AWK for good measure.</p>
<div class="highlight"><pre><span></span><code>gpg-connect-agent <span class="s1">'keyinfo --list'</span> /bye <span class="m">2</span>>/dev/null <span class="p">|</span> awk <span class="s1">'BEGIN{CACHED=0} /^S/ {match($0, /S\sKEYINFO\s\S+\s\S\s\S+\s\S+\s(\S)\s\S\s\S+\s\S+\s\S/, m); if(m[1]==1){CACHED=1}} END{print CACHED}'</span>
</code></pre></div>
<p><strong>EDIT 19FEB2017:</strong> In hind sight, there is a better awk, which now doesn't have to be gawk:</p>
<div class="highlight"><pre><span></span><code>gpg-connect-agent <span class="s1">'keyinfo --list'</span> /bye <span class="m">2</span>>/dev/null <span class="p">|</span> awk <span class="s1">'BEGIN{CACHED=0} /^S/ {if($7==1){CACHED=1}} END{if($0!=""){print CACHED} else {print "none"}}'</span>
</code></pre></div>
<h3 id="links">Links</h3>
<p><a href="https://demu.red/blog/2016/08/how-to-check-if-your-gpg-key-is-in-cache-part-2/">Part2</a><br/>
<a href="https://demu.red/blog/2017/03/how-to-check-if-your-smartcards-gpg-key-is-in-cache-part-3/">Part3</a><br/>
<a href="https://notabug.org/demure/dotfiles/src/master/i3/lemonbar">WM's bar</a><br/>
<a href="https://demu.red/blog/2016/05/backups-options-and-effort/">My Backups</a><br/>
<a href="https://alexcabal.com/creating-the-perfect-gpg-keypair/">Creating the Perfect GPG Keypair</a> </p>Backups, Options, and Effort2016-05-06T13:15:00-04:002016-05-06T13:15:00-04:00demuretag:demu.red,2016-05-06:/blog/2016/05/backups-options-and-effort/<p><strong>Followup <a href="https://demu.red/blog/2017/02/backups-revisited/">Backups Revisited</a> 10FEB2017</strong>
<strong>Followup <a href="https://demu.red/blog/2016/07/a-conky-display-for-my-backups/">Display</a></strong></p>
<h3 id="backups">Backups</h3>
<p>blah blah blah.
We all know, or should know, that backups are important.</p>
<h3 id="options">Options</h3>
<p>There are a <a href="https://wiki.archlinux.org/index.php/Synchronization_and_backup_programs">Lot of Options</a> out there.
A lot of them don't do what I want.
There are a few criteria that I desire, deltas, encrypted transfer, encrypted …</p><p><strong>Followup <a href="https://demu.red/blog/2017/02/backups-revisited/">Backups Revisited</a> 10FEB2017</strong>
<strong>Followup <a href="https://demu.red/blog/2016/07/a-conky-display-for-my-backups/">Display</a></strong></p>
<h3 id="backups">Backups</h3>
<p>blah blah blah.
We all know, or should know, that backups are important.</p>
<h3 id="options">Options</h3>
<p>There are a <a href="https://wiki.archlinux.org/index.php/Synchronization_and_backup_programs">Lot of Options</a> out there.
A lot of them don't do what I want.
There are a few criteria that I desire, deltas, encrypted transfer, encrypted storage, space conservation, remote backup.
There are few programs that did these, and did them in a way I liked.
<a href="http://rsnapshot.org/">rsnapshot</a> and <a href="http://obnam.org/">obnam</a> are the two I put most of my attention into.
I did make an exception for <code>rsnapshot</code> in that it does not do encryption itself, but luks+dmcrypt deals with that.</p>
<h4 id="rsnapshot">rsnapshot</h4>
<p>I started off playing with <code>rsnapshot</code> as it sounded very tasty, with its <a href="https://rsync.samba.org/">rsync</a> use, and log like rotation of backups, and hard linking files between backups.
To start with, I found <a href="https://pig-monkey.com/2012/09/cryptshot-automated-encrypted-backups-rsnapshot/">pigmonkey's post</a> about their use of <code>rsnapshot</code> (I've been subscribed to their blog for years, and hey! A post about what I'm trying to use!).
Their script (<a href="https://github.com/pigmonkey/backups/blob/master/cryptshot.sh">cryptshot</a>) worked great locally, though the first thing I did was to tweak it to gpg decrypt my luks keyfile.</p>
<div class="highlight"><pre><span></span><code>gpg2 --no-permission-warning --no-tty -qd <span class="nv">$KEYFILE</span> <span class="p">|</span> cryptsetup luksOpen --key-file<span class="o">=</span>- <span class="nv">$volume</span> <span class="nv">$name</span>
</code></pre></div>
<p><sup>NOTE: --no-permission-warning is there due to the use of <code>sudo -E</code>.</sup></p>
<p>This was all well and good, but I wanted backups to happen automatically.
Since I use a laptop, hooking in the external drive would be a thing that prevents this from being automagic.
So, I start tweaking the script, and make a 'backups' user on my pi, giving them <code>NOPASSWD</code> permissions in <code>/etc/sudoers</code> with exact augments provided, so that the account can't be missused.
I figure out that I can stream the gpg decrypted luks key via ssh as I only want it to live on my laptop, and have the process controlled by ssh-agent and gpg-agent being cached.</p>
<div class="highlight"><pre><span></span><code>gpg2 --no-permission-warning --no-tty -qd <span class="nv">$KEYFILE</span> <span class="p">|</span> ssh <span class="nv">$re_addr</span> <span class="s2">"sudo cryptsetup luksOpen --key-file=- </span><span class="nv">$volume</span><span class="s2"> </span><span class="nv">$name</span><span class="s2">"</span>
</code></pre></div>
<p>Then I actually started playing with <code>/etc/sudoers.d</code> as it has some benefits, and adjusting permissions.
I'm going to get to this section later, as I went through a number of revisions.</p>
<p>At this point I made a user on the backup server, and locked them down.
I'll go into specifics later.</p>
<p>So, this where I ran into an annoying wall.
<code>rsnapshot</code> only works on a 'pull' model.
This is fine if everything is local.
And also means you can pull an external server's content.
But not the other way around...</p>
<p>Which is exactly what I wanted to do, as my GPG key is staying on my computer and is only unlocked for a few hours.
I periodically reopen the key when I refresh mutt/pianobar, which would mean if-i-am-around -> do-the-backup.</p>
<p>So I set my <code>rsnapshot</code> plans down, and look at my options again.</p>
<h4 id="obnam">obnam</h4>
<p>For the next two days I poked <code>obnam</code> instead.
It seems nice, works on the push philosophy, does deltas, can use gpg encryption.</p>
<p>While trying it out I run into a few issues though.
<code>obnam</code> uses <code>gpg1</code>, or at the very least <code>/usr/bin/gpg</code>.
A future (soon) version of debian sid may change this, as I hear news of gpg2 being <code>/usr/bin/gpg</code> in the works. <sup>(has happened now, on debian sid 17AUG2016)</sup></p>
<p>Anyway, this is an 'issue', as my low-importance key (for things like <code>pass</code>, local files, etc) is a gpg2 key, and gpg1 just doesn't see it.</p>
<p>*<em>sigh</em>*</p>
<p>Next up: It takes longer to backup than <code>rsnapshot</code>.
I didn't do scientific testing here, but it really seemed to take 10-15 minutes to do a delta backup with <code>obnam</code>, where <code>rsnapshot</code> knocked it out in about 5-7 minutes.
(context: local backups with external plugged straight into laptop, total hard drive usage <300gb out of 500gb, external drive is 1tb)</p>
<p>The gpg issue isn't <code>obnam</code>'s fault, and the backup speed is just a little annoying.</p>
<p>The third issue was a deal breaker...
<code>obnam</code> is tar based, and offers two ways of accessing the backups, fuse and extraction.
The added layer of fuse makes unfortunately <strong><em>really</em></strong> slows down your interaction speed.
If I run <a href="https://dev.yorhel.nl/ncdu">ncdu</a> on my laptop, it takes ~5 minutes to the scan.
If I have one backup via <code>rsnapshot</code> it takes ~5 minutes to scan (or if I just point <code>ncdu</code> at one hourly backup).
If I have one backup in <code>obnam</code> it takes >3 hours to <code>ncdu</code>... <strong><em>ffs!</em></strong></p>
<p>So, back to the drawing board.</p>
<h4 id="back to rsnapshot">Back to rsnapshot</h4>
<p>At this point I look at <code>rsnapshot</code> again.
I mostly put it down, as thinking about the threat model of letting the backup server have access to my laptop was going to take effort.</p>
<p>I decide I will have to give the backup user limited access to my laptop, if I want backups to work and be automatic.
There will be a backup user on both the pi and the laptop, passwords will be disabled with <code>passwd -l</code> which still allows key based ssh login.
The ssh key will have restricted IPs, and point at a script that checks the incoming commands (even though sudo checks too).</p>
<h3 id="end setup_1">End Setup</h3>
<p>Here is a summary of how I tied everything together.</p>
<h4 id="users">Users</h4>
<p>Both the laptop and the server have a backup user, with their password disabled</p>
<div class="highlight"><pre><span></span><code><span class="c1"># Add the user</span>
sudo adduser backups
<span class="c1"># Become the user</span>
sudo su - backups
<span class="c1"># I'm lazy, this makes ~/.ssh with the right premissions.</span>
ssh localhost
<span class="nb">exit</span>
<span class="c1"># make a non standard key</span>
ssh-keygen -b BIG_NUMBER_HERE
<span class="nb">exit</span>
<span class="c1"># Disable the user password</span>
sudo passwd -l backups
</code></pre></div>
<p><sup>NOTE: Non standard key sizes (and primes) are a good thing!<sup></sup></sup></p>
<h4 id="ssh">SSH</h4>
<p>Installed to <code>/usr/local/sbin/val-back-cmd.sh</code> for permission security. </p>
<p>These accounts have an <code>authorized_keys</code> file.
The keys are restricted to set IPs, and run a script that checks the incoming commands.</p>
<div class="highlight"><pre><span></span><code><span class="c1">#~/.ssh/authorized_keys</span>
<span class="nv">from</span><span class="o">=</span><span class="s2">"10.0.0.2,10.0.0.3"</span>,command<span class="o">=</span><span class="s2">"/usr/local/sbin/val-back-cmd.sh"</span> SSH_KEY_HERE
</code></pre></div>
<p>Paired with:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#! /bin/bash</span>
<span class="nv">laptop_host</span><span class="o">=</span><span class="s2">"moving-computer-of-doom"</span>
<span class="nv">remote_host</span><span class="o">=</span><span class="s2">"tiny-server-of-doom"</span>
<span class="k">if</span> <span class="o">[</span> <span class="nv">$HOSTNAME</span> <span class="o">=</span> <span class="nv">$laptop_host</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>
<span class="k">case</span> <span class="s2">"</span><span class="nv">$SSH_ORIGINAL_COMMAND</span><span class="s2">"</span> in
*<span class="se">\&</span>*<span class="p">|</span>*<span class="se">\|</span>*<span class="p">|</span>*<span class="se">\;</span>*<span class="p">|</span>*<span class="se">\></span>*<span class="p">|</span>*<span class="se">\<</span>*<span class="p">|</span>*<span class="se">\!</span>*<span class="o">)</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="p">;;</span>
<span class="c1">## This needs to be set in /etc/rsnapshot.conf rsync_long_args</span>
/usr/bin/rsync<span class="se">\ </span>--server<span class="se">\ </span>--sender*<span class="o">)</span>
sudo <span class="nv">$SSH_ORIGINAL_COMMAND</span>
<span class="p">;;</span>
*<span class="o">)</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="p">;;</span>
<span class="k">esac</span>
<span class="k">elif</span> <span class="o">[</span> <span class="nv">$HOSTNAME</span> <span class="o">=</span> <span class="nv">$remote_host</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>
<span class="k">case</span> <span class="s2">"</span><span class="nv">$SSH_ORIGINAL_COMMAND</span><span class="s2">"</span> in
*<span class="se">\&</span>*<span class="p">|</span>*<span class="se">\|</span>*<span class="p">|</span>*<span class="se">\;</span>*<span class="p">|</span>*<span class="se">\></span>*<span class="p">|</span>*<span class="se">\<</span>*<span class="p">|</span>*<span class="se">\!</span>*<span class="o">)</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="p">;;</span>
mount*<span class="o">)</span>
sudo <span class="nv">$SSH_ORIGINAL_COMMAND</span>
<span class="p">;;</span>
umount*<span class="o">)</span>
sudo <span class="nv">$SSH_ORIGINAL_COMMAND</span>
<span class="p">;;</span>
cryptsetup<span class="se">\ </span>luksOpen*<span class="o">)</span>
sudo <span class="nv">$SSH_ORIGINAL_COMMAND</span>
<span class="p">;;</span>
cryptsetup<span class="se">\ </span>luksClose*<span class="o">)</span>
sudo <span class="nv">$SSH_ORIGINAL_COMMAND</span>
rmdir*<span class="o">)</span>
sudo <span class="nv">$SSH_ORIGINAL_COMMAND</span>
<span class="p">;;</span>
mkdir*<span class="o">)</span>
sudo <span class="nv">$SSH_ORIGINAL_COMMAND</span>
<span class="p">;;</span>
rsnapshot*<span class="o">)</span>
sudo <span class="nv">$SSH_ORIGINAL_COMMAND</span>
<span class="p">;;</span>
*<span class="o">)</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="p">;;</span>
<span class="k">esac</span>
<span class="k">fi</span>
</code></pre></div>
<p>It is worth noting that you do need to manually connect from laptop-to-server, and server-to-laptop at least once, so that the host key is accepted.</p>
<p><a href="https://notabug.org/demure/cryptshotr/src/master/val-back-cmd.sh">Example val-back-cmd.sh</a></p>
<h4 id="sudoers">Sudoers</h4>
<p>Eariler I talked about setting up <code>/etc/sudoers</code>.</p>
<p>But thinking about it, lets set this up better, and use <code>/etc/sudoers.d/</code>.
Now we don't have to worry about the order, and not backups will not be effected by future <code>sudo</code> package updates.
To make a new entry, <code>sudo visudo -f /etc/sudoers.d/backup_sudo</code>.</p>
<div class="highlight"><pre><span></span><code>Cmnd_Alias <span class="nv">MOUNT</span> <span class="o">=</span> /bin/mount /dev/mapper/crypt-YOUR_UUID_HERE /mnt/YOUR_UUID_HERE, /bin/umount /mnt/YOUR_UUID_HERE
Cmnd_Alias <span class="nv">LUKS</span> <span class="o">=</span> /sbin/cryptsetup luksOpen --key-file<span class="se">\=</span>- /dev/disk/by-uuid/YOUR_UUID_HERE crypt-YOUR_UUID_HERE, /sbin/cryptsetup luksClose crypt-YOUR_UUID_HERE
Cmnd_Alias <span class="nv">DIR</span> <span class="o">=</span> /bin/rmdir /mnt/YOUR_UUID_HERE, /bin/mkdir /mnt/YOUR_UUID_HERE
Cmnd_Alias <span class="nv">RSYNC</span> <span class="o">=</span> /usr/bin/rsync
Cmnd_Alias <span class="nv">RSNAPSHOT</span> <span class="o">=</span> /usr/bin/rsnapshot hourly, /usr/bin/rsnapshot daily, /usr/bin/rsnapshot weekly, /usr/bin/rsnapshot monthly, /usr/bin/rsnapshot yearly
Cmnd_Alias <span class="nv">CRYPTSHOTR</span> <span class="o">=</span> /usr/local/sbin/cryptshotr -q hourly, /usr/local/sbin/cryptshotr -q daily, /usr/local/sbin/cryptshotr -q weekly, /usr/local/sbin/cryptshotr -q monthly, /usr/local/sbin/cryptshotr -q yearly
<span class="c1">## On laptop, when remote is accessing</span>
<span class="c1">## Laptop needs RSYNC and CRYPTSHOTR, as 'sudo cryptshotr' will handle the decrypting/mounting/mkdir. laptop backup user needs RSYNC. 'cryptshotr-cron' needs CRYPTSHOTR.</span>
<span class="c1">## Remote server needs MOUNT, LUKS, DIR, RSYNC, and RSNAPSHOT.</span>
%backups <span class="nv">ALL</span><span class="o">=(</span>ALL<span class="o">)</span> NOPASSWD: RSYNC, CRYPTSHOTR
</code></pre></div>
<p>With the commands broken up by type, it makes it much more readable, and I don't need to give the backup server CRYPTSHOTR access, etc.</p>
<p><a href="https://notabug.org/demure/cryptshotr/src/master/backup_sudoers">Example backup_sudoers</a></p>
<h4 id="rsnapshot_1">rsnapshot</h4>
<p>Installed to <code>/etc/rsnapshot.conf</code> </p>
<p><strong><em>TABS</em></strong>, you must use tabs.
Really, options and augments have to be tab separated.
I've warned you.</p>
<p>There are three big settings, as well as the <code>interval</code> and <code>exclude</code> section.
The follow is just to highlight those settings, it is incomplete.</p>
<div class="highlight"><pre><span></span><code>snapshot_root /mnt/YOUR_UUID_HERE
<span class="c1">#...</span>
<span class="c1">### Intervals ### {{{</span>
<span class="c1">## Must be unique and in ascending order</span>
retain hourly <span class="m">24</span>
retain daily <span class="m">7</span>
retain weekly <span class="m">4</span>
retain monthly <span class="m">12</span>
retain yearly <span class="m">5</span>
<span class="c1">### End Intercals ### }}}</span>
<span class="c1">#...</span>
rsync_long_args -ev --rsync-path<span class="o">=</span>/usr/bin/rsync
<span class="c1">#...</span>
<span class="c1">## For local cryptshotr</span>
backup / doom/
<span class="c1">## For remote cryptshotr</span>
<span class="c1">#backup backups@moving-computer-of-doom.local:/ doom/</span>
</code></pre></div>
<p><a href="https://notabug.org/demure/cryptshotr/src/master/rsnapshot.conf">Example rsnapshot</a></p>
<h4 id="cryptshotr">cryptshotr</h4>
<p>Installed to <code>/usr/local/sbin/cryptshotr</code> for permission security. </p>
<p>For <code>cryptshotr</code>, my modification of <code>cryptshot</code> by pigmonkey, I ended up doing a few things.
I already talked about using a GPG encrypted LUKs keyfile.
The rest of my changes are going to be summarized here.</p>
<ul>
<li>☑ Uses GPG with LUKs key, so it is secure at rest.<ul>
<li>GPG keys are cached with gpg-agent, which I periodically refresh for mutt/pianobar.</li>
</ul>
</li>
<li>☑ Can backup over the network.<ul>
<li>Checks if remote feature is enabled in settings.</li>
<li>Checks if connected to home network, via MAC address.</li>
<li>Checks if remote server is pingable.</li>
<li>Checks if remote volume is connected.</li>
<li>Will then pipe gpg decrypted LUKs key over ssh, to open volume; then mount container.</li>
<li>This works via my Pi at home</li>
</ul>
</li>
<li>☑ Added a 'quiet' mode for less crontab emails<ul>
<li>I expect a number of backups to fail:<ul>
<li>My laptop is moble</li>
<li>My gpg-agent will time out</li>
<li>My server might not recover from the router rebooting <sub>(working on that)</sub></li>
</ul>
</li>
</ul>
</li>
</ul>
<p><a href="https://notabug.org/demure/cryptshotr/src/master/cryptshotr">cryptshotr code</a></p>
<h4 id="cryptshotr-cron">cryptshotr-cron</h4>
<p>Installed to <code>/usr/local/sbin/cryptshotr-cron</code> for permission security. </p>
<p>I wrote a wrapper to deal with schduling run times, and running periodicities in order.
<code>crontab</code> would have ran multiple periodicities in parallel if they were scheduled at the same time.
This would cause all besides the first to fail, as the container would already be open.
While I considered teaching the script to not care, I decided that leaving it as is would be more tamper-evident.</p>
<ul>
<li>☑ log file<ul>
<li>Will log periodicities' pass/fail</li>
<li>On pass, will purge previous entries for periodicity</li>
</ul>
</li>
<li>☑ Will check periodicity's last run epoch against current time<ul>
<li>If time not met, skips periodicity.</li>
<li>If missing runs periodicity.</li>
<li>If last run failed, run.</li>
</ul>
</li>
</ul>
<p><a href="https://notabug.org/demure/cryptshotr/src/master/cryptshotr-cron">cryptshotr-cron code</a></p>
<h4 id="crontab">crontab</h4>
<p>To finish it all up, I call cryptshotr-cron in crontab every half hour.</p>
<div class="highlight"><pre><span></span><code><span class="x">*/30 * * * * /usr/local/sbin/cryptshotr-cron</span>
</code></pre></div>
<h3 id="my git repo_1">My git repo</h3>
<p>I have set up a <strong><a href="https://notabug.org/demure/cryptshotr">git repo</a></strong> with the files in one place, if people wish to use my work (built off of pigmonkey's).</p>
<h3 id="todo">TODO</h3>
<ul>
<li>Make script for server to detect orphaned volume mount (backup failed/hung/laptop closed), and shut the container.</li>
</ul>
<h3 id="links">Links</h3>
<p><a href="https://demu.red/blog/2017/02/backups-revisited/">Backups Revisited</a><br/>
<a href="https://demu.red/blog/2016/07/a-conky-display-for-my-backups/">Display</a><br/>
<a href="https://wiki.archlinux.org/index.php/Synchronization_and_backup_programs">Lot of Options</a><br/>
<a href="http://rsnapshot.org/">rsnapshot</a><br/>
<a href="http://obnam.org/">obnam</a><br/>
<a href="https://rsync.samba.org/">rsync</a><br/>
<a href="https://pig-monkey.com/2012/09/cryptshot-automated-encrypted-backups-rsnapshot/">pigmonkey's post</a><br/>
<a href="https://github.com/pigmonkey/backups/blob/master/cryptshot.sh">cryptshot</a><br/>
<a href="https://dev.yorhel.nl/ncdu">ncdu</a><br/>
<a href="https://notabug.org/demure/cryptshotr/src/master/rsnapshot.conf">Example rsnapshot</a><br/>
<a href="https://notabug.org/demure/cryptshotr/src/master/cryptshotr">cryptshotr code</a><br/>
<a href="https://notabug.org/demure/cryptshotr/src/master/cryptshotr-cron">cryptshotr-cron code</a><br/>
<a href="https://notabug.org/demure/cryptshotr">git repo</a> </p>
<p><cr><sub><strong>Edited 18MAY2016</strong>: Improved val script's cryptsetup slightly, suggested by pigmonkey.</sub></cr></p>Setting up wicd for WPA2 PEAP without a domain field2016-03-17T17:16:00-04:002016-03-17T17:16:00-04:00demuretag:demu.red,2016-03-17:/blog/2016/03/setting-up-wicd-for-wpa2-peap-without-a-domain-field/<h3 id="legacy setup">Legacy Setup</h3>
<p>Until recently, I had relied on wifi settings cached by <code>nm-applet</code>.
I had already started using <code>wicd-curses</code> months ago, after killing almost everything gnome related from my install.
When my laptop stopped auto connecting to known access points, I got around to becoming familiar with <a href="http://wicd.sourceforge.net/">wicd</a>'s interface …</p><h3 id="legacy setup">Legacy Setup</h3>
<p>Until recently, I had relied on wifi settings cached by <code>nm-applet</code>.
I had already started using <code>wicd-curses</code> months ago, after killing almost everything gnome related from my install.
When my laptop stopped auto connecting to known access points, I got around to becoming familiar with <a href="http://wicd.sourceforge.net/">wicd</a>'s interface.</p>
<h3 id="networks">Networks</h3>
<p>At my college, I had a bit of an issue.
There are two wifi connections at school.
One is a crappy captive portal based login, that seems to force you to login every five minutes.
It also tends to be broken at the start of every semester, due to all the new people trying to get on it.</p>
<p>The second connection is the NAME-secure wifi, that you must connect to using your school credentials.
This one works well, once you find the magic℠ (undocumented) settings.
I assume the IT department doesn't realize the need for this documentation, as windblows/android/mac/ios can guess the magic℠ settings on their own.
Even when I was using <code>nm-applet</code>, I had to annoy the IT people into telling me the right combination of settings...</p>
<h3 id="wicd">wicd</h3>
<p>Anyway, having those settings in the <code>nm-applet</code> conf files gave me a head start this time.
I needed to use WPA2 PEAP, with MSCHAPv2.
So I bring up <code>wicd-curses</code>, go to the connection settings, toggle on 'autoconnect' and 'use encryption' and 'Use these settings for all networks sharing this essid' (as there are many of these throughout different buildings).
At this point, I check my encryption options, and there is 'WPA2-PEAP with CCMP/MSCHAPV2'!
Great, this will be easy!
Err... nevermind.</p>
<p>It asks for a 'domain' field, something the magic℠ settings don't use...</p>
<h4 id="wicd templets">wicd Templets</h4>
<p>At this point, I learn how <code>wicd</code> works, and find that it has templates.
In <code>/etc/wicd/encryption/templates</code> I find that is <code>/etc/wicd/encryption/templates/wpa2-peap</code>is 'WPA2-PEAP with CCMP/MSCHAPV2' and read through it.</p>
<div class="highlight"><pre><span></span><code><span class="nv">name</span> <span class="o">=</span> WPA2-PEAP with CCMP/MSCHAPV2
<span class="nv">author</span> <span class="o">=</span> atiketemola
<span class="nv">version</span> <span class="o">=</span> <span class="m">1</span>
require identity *Username domain *Domain password *Password
protected password *Password
-----
<span class="nv">ctrl_interface</span><span class="o">=</span>/var/run/wpa_supplicant
<span class="nv">network</span><span class="o">={</span>
<span class="nv">ssid</span><span class="o">=</span><span class="s2">"</span><span class="nv">$_ESSID</span><span class="s2">"</span>
<span class="nv">proto</span><span class="o">=</span>RSN
<span class="nv">key_mgmt</span><span class="o">=</span>WPA-EAP
<span class="nv">pairwise</span><span class="o">=</span>CCMP
<span class="nv">eap</span><span class="o">=</span>PEAP
<span class="nv">identity</span><span class="o">=</span><span class="s2">"</span><span class="nv">$_DOMAIN</span><span class="s2">\$_IDENTITY"</span>
<span class="nv">password</span><span class="o">=</span><span class="s2">"</span><span class="nv">$_PASSWORD</span><span class="s2">"</span>
<span class="nv">phase2</span><span class="o">=</span><span class="s2">"auth=MSCHAPv2"</span>
<span class="o">}</span>
</code></pre></div>
<p>Simple enough fix, I copy it to <code>/etc/wicd/encryption/templetes/wpa2-peap-nodomain</code> and edit:</p>
<div class="highlight"><pre><span></span><code><span class="nv">name</span> <span class="o">=</span> WPA2 PEAP No-Domain
<span class="nv">author</span> <span class="o">=</span> demure
<span class="nv">version</span> <span class="o">=</span> <span class="m">1</span>
require identity *Username password *Password
protected password *Password
-----
<span class="nv">ctrl_interface</span><span class="o">=</span>/var/run/wpa_supplicant
<span class="nv">network</span><span class="o">={</span>
<span class="nv">ssid</span><span class="o">=</span><span class="s2">"</span><span class="nv">$_ESSID</span><span class="s2">"</span>
<span class="nv">proto</span><span class="o">=</span>RSN
<span class="nv">key_mgmt</span><span class="o">=</span>WPA-EAP
<span class="nv">pairwise</span><span class="o">=</span>CCMP
<span class="nv">eap</span><span class="o">=</span>PEAP
<span class="nv">identity</span><span class="o">=</span><span class="s2">"</span><span class="nv">$_IDENTITY</span><span class="s2">"</span>
<span class="nv">password</span><span class="o">=</span><span class="s2">"</span><span class="nv">$_PASSWORD</span><span class="s2">"</span>
<span class="nv">phase2</span><span class="o">=</span><span class="s2">"auth=MSCHAPv2"</span>
<span class="o">}</span>
</code></pre></div>
<p>Perfect, now it should work!</p>
<p>Err, maybe not...</p>
<p>The template part of the man page doesn't mention it, but you need to edit <code>/etc/wicd/encryption/templates/active</code> for new templates to show up.</p>
<p>While I'm at it, why don't I just put WPA2 options at the top.</p>
<div class="highlight"><pre><span></span><code>wpa2-leap
wpa2-peap
wpa2-peap-nodomain
wpa
wpa-peap
wpa-psk
wpa-psk-hex
wep-hex
wep-passphrase
wep-shared
leap
ttls
eap
peap
peap-tkip
eap-tls
psu
</code></pre></div>
<p>And <em>now</em> there is wifi.</p>
<h3 id="links_1">Links</h3>
<p><a href="http://wicd.sourceforge.net/">wicd</a></p>i3, lemonbar, urxvt, and touchegg2016-03-16T21:59:00-04:002016-03-16T21:59:00-04:00demuretag:demu.red,2016-03-16:/blog/2016/03/i3-lemonbar-urxvt-and-touchegg/<p><strong>NOTE:</strong> I've moved away from lemonbar and have reimplmented my bar in polybar, works much better. You can see my polybar conf in my dotfiles.</p>
<h3 id="window manger">Window Manger</h3>
<h4 id="spectrwm">spectrwm</h4>
<p>Until recently, I was using <a href="https://github.com/conformal/spectrwm/wiki">spectrwm</a> for my Window Manager (WM).
When I first switched from a non tiling WM to spectrwm …</p><p><strong>NOTE:</strong> I've moved away from lemonbar and have reimplmented my bar in polybar, works much better. You can see my polybar conf in my dotfiles.</p>
<h3 id="window manger">Window Manger</h3>
<h4 id="spectrwm">spectrwm</h4>
<p>Until recently, I was using <a href="https://github.com/conformal/spectrwm/wiki">spectrwm</a> for my Window Manager (WM).
When I first switched from a non tiling WM to spectrwm, I liked it for a lot of reasons.
Overall it reminded me a great deal of <a href="https://tmux.github.io/">tmux</a>, and tmux really made sense to me.
A few of my friends often told me now nice <a href="https://i3wm.org/">i3</a> was, and I poked it a few times.</p>
<p>I never really had motivation to leave spectrwm, but after a while two things were annoying me.
There were always a few programs that just didn't want to work with spectrwm.
The most obvious examples are various steam games... (while I try to use FOSS, I do have some weak points)
Another issue that annoyed me was that <a href="https://github.com/brndnmtthws/conky">conky</a> via spectrwm's bar could only do one font.
There are very few good unicode fonts that exist, and fewer still that do mono space well.
Even <a href="http://www.fsd.it/shop/fonts/pragmatapro/">Pragmata Pro</a> has room to improve.
Of the fonts that do exist, non xft font handling by programs is soooo annoying.
Anyway, I used unicode to reduce horizontal bar space, but it looked meh due to many issues.</p>
<h4 id="i3">i3</h4>
<p>So, I came across a very nice looking bar configuration in i3, using lemonbar, <a href="https://github.com/electro7/dotfiles/tree/master/.i3/lemonbar">electro7's i3 lemonbar</a>.
(Due to non xft hijinks, I recommend <a href="https://github.com/krypt-n/bar">krypt-n's lemonbar</a> instead of normal lemonbar.)
This beauty gave me some inspiration to migrate to i3.
So, I started my move by tinkering with electro7's work, adding things I wanted, and in a few places making the code work better (it used to eat up space for both eth0 and wlan0 up/down speed, even if one was inactive).
Aside from tinkering with having control-pianobar (and/or cmus) output, I think <a href="https://notabug.org/demure/dotfiles/src/master/i3/lemonbar">my mod</a> is good till I get a bigger monitor.</p>
<h5 id="modes">Modes</h5>
<p>After my new fancy bar was good, I sat down and started configuring i3 itself.
Priorities, right?
While setting i3 up, I found it can do some fun things with <code>modes</code>.</p>
<div class="highlight"><pre><span></span><code><span class="c1">### Handy Menu Mode ### {{{</span>
<span class="nb">set</span> <span class="nv">$mode_system</span> System <span class="o">(</span>l<span class="o">)</span> lock, <span class="o">(</span>e<span class="o">)</span> logout, <span class="o">(</span>s<span class="o">)</span> suspend, <span class="o">(</span>h<span class="o">)</span> hibernate, <span class="o">(</span>r<span class="o">)</span> reboot, <span class="o">(</span>Shift+s<span class="o">)</span> shutdown
mode <span class="s2">"</span><span class="nv">$mode_system</span><span class="s2">"</span> <span class="o">{</span>
bindsym l <span class="nb">exec</span> --no-startup-id <span class="nv">$Locker</span>, mode <span class="s2">"default"</span>
bindsym e <span class="nb">exec</span> --no-startup-id i3-msg exit, mode <span class="s2">"default"</span>
bindsym s <span class="nb">exec</span> --no-startup-id <span class="nv">$Locker</span> <span class="o">&&</span> systemctl suspend, mode <span class="s2">"default"</span>
bindsym h <span class="nb">exec</span> --no-startup-id <span class="nv">$Locker</span> <span class="o">&&</span> systemctl hibernate, mode <span class="s2">"default"</span>
bindsym r <span class="nb">exec</span> --no-startup-id systemctl reboot, mode <span class="s2">"default"</span>
bindsym Shift+s <span class="nb">exec</span> --no-startup-id systemctl poweroff -i, mode <span class="s2">"default"</span>
<span class="c1"># back to normal: Enter or Escape</span>
bindsym Return mode <span class="s2">"default"</span>
bindsym Escape mode <span class="s2">"default"</span>
<span class="o">}</span>
bindsym <span class="nv">$mod</span>+Pause mode <span class="s2">"</span><span class="nv">$mode_system</span><span class="s2">"</span>
<span class="c1">### End Handy Menu Mode ### }}}</span>
<span class="c1">### control-pianobar Mode ### {{{</span>
mode <span class="s2">"control-pianobar"</span> <span class="o">{</span>
bindsym p <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh play, mode <span class="s2">"default"</span>
bindsym q <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh quit, mode <span class="s2">"default"</span>
bindsym h <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh history, mode <span class="s2">"default"</span>
bindsym n <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh next, mode <span class="s2">"default"</span>
bindsym t <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh tired, mode <span class="s2">"default"</span>
bindsym s <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh switchstation, mode <span class="s2">"default"</span>
bindsym <span class="se">\,</span> <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh previousstation, mode <span class="s2">"default"</span>
bindsym . <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh nextstation, mode <span class="s2">"default"</span>
bindsym l <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh love, mode <span class="s2">"default"</span>
bindsym b <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh ban, mode <span class="s2">"default"</span>
bindsym e <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh explain, mode <span class="s2">"default"</span>
bindsym c <span class="nb">exec</span> --no-startup-id ~/.config/pianobar/control-pianobar.sh current, mode <span class="s2">"default"</span>
<span class="c1"># back to normal: Enter or Escape</span>
bindsym Return mode <span class="s2">"default"</span>
bindsym Escape mode <span class="s2">"default"</span>
<span class="o">}</span>
bindsym <span class="nv">$mod</span>+m mode <span class="s2">"control-pianobar"</span>
<span class="c1">### End control-pianobar Mode ### }}}</span>
</code></pre></div>
<p>Modes let you have completely different bindings then you normally have.
You just have to be careful to revert back to the default mode after a command, or build escape buttons in.</p>
<h3 id="other digressions, err... programs_2">Other Digressions, err... Programs</h3>
<h4 id="urxvt">urxvt</h4>
<p>In the middle of getting i3 set up, I let the same friends make me think about <a href="http://software.schmorp.de/pkg/rxvt-unicode.html">urxvt</a>, which I also find I like.
I really wasn't using <code>terminator</code> for much besides its color customization.
One of the first things I kill on terminator is its red bar for groups/splits, as I use tmux inside it.</p>
<h4 id="uzbl">uzbl</h4>
<p><a href="http://www.uzbl.org/">uzbl</a> seems like a nice, lightweight GUI web browser, with vim like bindings.
I am using it more and more for random look ups.
I helped encourage myself to use it by getting <a href="https://github.com/carnager/rofi-pass">rofi-pass</a> to work, so I can auto fill my passwords.</p>
<h3 id="reason i started writing this post_1">Reason I Started Writing This Post</h3>
<p>Having recently switched from spectrwm to i3 for my window manager, I have found that there are one or two things that you just can't do with the mouse.
For me, this is mainly switching between workspaces.
(Sure, this is true of spectrwm as well, but this fix can be used there too.)
After a few days of trying to find a method to do this, I came across <a href="https://github.com/JoseExposito/touchegg">touchegg</a>.</p>
<blockquote>
<p>Touchegg is a multitouch gesture program, that runs as a user in the background, and adds multitouch support to window managers.
-arch wiki</p>
</blockquote>
<p>Installing touchegg is pretty simple, but its instructions are biased towards ubuntu.
touchegg tells you to install <code>build-essential libqt4-dev utouch libgeis-dev libx11-6 libxtst-dev</code>; on debian sid <code>utouch</code> is acually <code>libgeis-dev</code>.</p>
<p>Once it is installed, running <code>touchegg</code> will put the default config in place.
It will also display some helpful suggestions for testing, and display your actions for debug.
The manual, only found on touchegg's github wiki, suggests a number of synclient settings be turned off:</p>
<div class="highlight"><pre><span></span><code><span class="x">synclient TapButton2=0</span>
<span class="x">synclient TapButton3=0</span>
<span class="x">synclient ClickFinger2=0</span>
<span class="x">synclient ClickFinger3=0</span>
<span class="x">synclient HorizTwoFingerScroll=0</span>
<span class="x">synclient VertTwoFingerScroll=0</span>
</code></pre></div>
<p>Once you do this most of the gestures will work...
Except for pinching, which segfaults.
After testing the available options, I decided four finger swipes are all I need for now, so I reverted my synclient changes.</p>
<h5 id="conf">Conf</h5>
<p>At this point I just added a few lines to (and deleted unused defaults from) my <code>~/.config/touchegg/touchegg.conf</code> </p>
<div class="highlight"><pre><span></span><code><span class="nt"><application</span> <span class="na">name=</span><span class="s">"All"</span><span class="nt">></span>
<span class="nt"><gesture</span> <span class="na">type=</span><span class="s">"DRAG"</span> <span class="na">fingers=</span><span class="s">"4"</span> <span class="na">direction=</span><span class="s">"LEFT"</span><span class="nt">></span>
<span class="nt"><action</span> <span class="na">type=</span><span class="s">"SEND_KEYS"</span><span class="nt">></span>Super+U<span class="nt"></action></span>
<span class="nt"></gesture></span>
<span class="nt"><gesture</span> <span class="na">type=</span><span class="s">"DRAG"</span> <span class="na">fingers=</span><span class="s">"4"</span> <span class="na">direction=</span><span class="s">"RIGHT"</span><span class="nt">></span>
<span class="nt"><action</span> <span class="na">type=</span><span class="s">"SEND_KEYS"</span><span class="nt">></span>Super+O<span class="nt"></action></span>
<span class="nt"></gesture></span>
<span class="nt"></application></span>
</code></pre></div>
<p>As well as my <code>~/.i3/config</code>.</p>
<div class="highlight"><pre><span></span><code><span class="c1">### Cycle Through Workspaces ### {{{</span>
<span class="c1">## Added for touchegg</span>
bindsym <span class="nv">$mod</span>+u workspace prev
bindsym <span class="nv">$mod</span>+o workspace next
<span class="c1">### End Cycle Through Workspaces ### }}}</span>
</code></pre></div>
<h3 id="links_1">Links</h3>
<p><a href="https://github.com/conformal/spectrwm/wiki">spectrwm</a><br/>
<a href="https://tmux.github.io/">tmux</a><br/>
<a href="https://i3wm.org/">i3</a><br/>
<a href="https://github.com/brndnmtthws/conky">conky</a><br/>
<a href="http://www.fsd.it/shop/fonts/pragmatapro/">Pragmata Pro</a><br/>
<a href="https://github.com/electro7/dotfiles/tree/master/.i3/lemonbar">electro7's i3 lemonbar</a><br/>
<a href="https://github.com/krypt-n/bar">krypt-n's lemonbar</a><br/>
<a href="https://notabug.org/demure/dotfiles/src/master/i3/lemonbar">my mod</a><br/>
<a href="http://software.schmorp.de/pkg/rxvt-unicode.html">urxvt</a><br/>
<a href="http://www.uzbl.org/">uzbl</a><br/>
<a href="https://github.com/carnager/rofi-pass">rofi-pass</a><br/>
<a href="https://github.com/JoseExposito/touchegg">touchegg</a> </p>VMware 12 for linux, and the ports it opens2016-03-16T21:39:00-04:002016-03-16T21:39:00-04:00demuretag:demu.red,2016-03-16:/blog/2016/03/vmware-12-for-linux-and-the-ports-it-opens/<h3 id="vmware">VMware</h3>
<p>I would be using <a href="http://wiki.qemu.org/Main_Page">qemu-kvm</a> by choice, but school is using VMware...
Rather than provide IT support for myself if I had a networking issue/etc, I chose to install VMware.
And at least I don't have to pay for it, as school provides a key; I would likely …</p><h3 id="vmware">VMware</h3>
<p>I would be using <a href="http://wiki.qemu.org/Main_Page">qemu-kvm</a> by choice, but school is using VMware...
Rather than provide IT support for myself if I had a networking issue/etc, I chose to install VMware.
And at least I don't have to pay for it, as school provides a key; I would likely have just used Qemu otherwise.
While school is on workstation 11, 11 flatly fails to work on current linux.
I could not force it to work on debian despite performing a few arcane rituals, though I could at least make it respond a bit more.
A classmate had the same issue on fedora.
So, I went and installed workstation 12 instead, and it-just-works™.</p>
<h3 id="the issue">The Issue</h3>
<p>It likes to open two ports by default, and it doesn't really tell you it is doing so...</p>
<!--
:::bash
~ -> nmap -sV localhost
Starting Nmap 7.01 ( https://nmap.org ) at 2016-03-16 17:44 EDT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00012s latency).
Other addresses for localhost (not scanned): ::1
Not shown: 995 closed ports
PORT STATE SERVICE VERSION
22/tcp open ssh OpenSSH 7.2p2 Debian 1 (protocol 2.0)
25/tcp open smtp Exim smtpd 4.86_2
443/tcp open ssl/http VMware VirtualCenter Web service
902/tcp open ssl/vmware-auth VMware Authentication Daemon 1.10 (Uses VNC, SOAP)
5432/tcp open postgresql PostgreSQL DB 9.3.3 - 9.3.5
Service Info: Host: moving-computer-of-doom.local; OS: Linux; CPE: cpe:/o:linux:linux_kernel
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 13.38 seconds
~ ->
-->
<div class="highlight"><pre><span></span><code>~ -> nmap -sV <span class="m">10</span>.0.0.3
Starting Nmap <span class="m">7</span>.01 <span class="o">(</span> https://nmap.org <span class="o">)</span> at <span class="m">2016</span>-03-16 <span class="m">17</span>:44 EDT
Nmap scan report <span class="k">for</span> doom.lan <span class="o">(</span><span class="m">10</span>.0.0.3<span class="o">)</span>
Host is up <span class="o">(</span><span class="m">0</span>.00013s latency<span class="o">)</span>.
Not shown: <span class="m">997</span> closed ports
PORT STATE SERVICE VERSION
<span class="m">22</span>/tcp open ssh OpenSSH <span class="m">7</span>.2p2 Debian <span class="m">1</span> <span class="o">(</span>protocol <span class="m">2</span>.0<span class="o">)</span>
<span class="m">443</span>/tcp open ssl/http VMware VirtualCenter Web service
<span class="m">902</span>/tcp open ssl/vmware-auth VMware Authentication Daemon <span class="m">1</span>.10 <span class="o">(</span>Uses VNC, SOAP<span class="o">)</span>
Service Info: OS: Linux<span class="p">;</span> CPE: cpe:/o:linux:linux_kernel
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap <span class="k">done</span>: <span class="m">1</span> IP address <span class="o">(</span><span class="m">1</span> host up<span class="o">)</span> scanned in <span class="m">13</span>.14 seconds
~ ->
</code></pre></div>
<p>So, port 443 and 902 are being put up by VMware...</p>
<h3 id="the fix">The Fix</h3>
<p>To deal with port 443, you will need to:</p>
<p>Menu -> Edit -> Preferences -> Shared VMs, untick "Enable virtual machine sharing and remote access".</p>
<p>To deal with port 902 is a bit harder. You will need to edit <code>/etc/init.d/vmware</code> and <code>/etc/init.d/vmware-workstation-server</code>.</p>
<p>For <code>/etc/init.d/vmware</code> comment out the four places containing "Authentication Daemon".</p>
<div class="highlight"><pre><span></span><code><span class="m">298</span> vmware_start_authdlauncher<span class="o">()</span> <span class="o">{</span>
<span class="m">299</span> <span class="c1">#vmware_bg_exec "`vmware_product_name` Authentication Daemon" \</span>
<span class="m">300</span> <span class="c1">#"$SBINDIR/vmware-authdlauncher"</span>
<span class="m">301</span> <span class="o">}</span>
~~~
<span class="m">348</span> <span class="c1">#vmware_exec 'VMware Authentication Daemon' vmware_start_authdlauncher</span>
~~~
<span class="m">360</span> stop<span class="o">)</span>
<span class="m">361</span> <span class="nb">echo</span> <span class="s1">'Stopping VMware services:'</span>
<span class="m">362</span> <span class="nv">exitcode</span><span class="o">=</span><span class="s1">'0'</span>¬
<span class="m">363</span>
<span class="m">364</span> <span class="c1">#vmware_exec 'VMware Authentication Daemon' vmware_stop_authdlauncher</span>
~~~
<span class="m">388</span> <span class="c1">#vmware_exec 'VMware Authentication Daemon' vmware_start_authdlauncher</span>
</code></pre></div>
<p>And for <code>/etc/init.d/vmware-workstation-server</code>:</p>
<div class="highlight"><pre><span></span><code><span class="m">16</span> <span class="c1">### END INIT INFO¬</span>
<span class="m">17</span> <span class="nb">echo</span> Aborting launch of vmware-workstation-server¬
<span class="m">18</span> <span class="nb">exit</span> <span class="m">0</span>¬
</code></pre></div>
<h3 id="links">Links</h3>
<p><a href="http://wiki.qemu.org/Main_Page">qemu-kvm</a></p>HOPM and your IRCd2016-01-06T17:08:00-05:002016-01-06T17:08:00-05:00demuretag:demu.red,2016-01-06:/blog/2016/01/hopm-and-your-ircd/<p><strong>NOTE 2020-05-25: Rather than use HOPM these days, I would point you at a modern IRCd like <a href="https://www.inspircd.org/">InspIRCd 3</a></strong></p>
<h3 id="intro">Intro</h3>
<p>I am making this post because it seems like <a href="https://github.com/ircd-hybrid/hopm">HOPM</a> is poorly advertised as a successor to <a href="https://github.com/azzurra/bopm">BOPM</a> on the internet.
As it took me more time to find HOPM …</p><p><strong>NOTE 2020-05-25: Rather than use HOPM these days, I would point you at a modern IRCd like <a href="https://www.inspircd.org/">InspIRCd 3</a></strong></p>
<h3 id="intro">Intro</h3>
<p>I am making this post because it seems like <a href="https://github.com/ircd-hybrid/hopm">HOPM</a> is poorly advertised as a successor to <a href="https://github.com/azzurra/bopm">BOPM</a> on the internet.
As it took me more time to find HOPM than it could have, I feel talking about it here may contribute to its search ranking (there are plenty of crawlers on my tiny site already).
The inspiration for this post is a recent botnet attack on my server, which seems to have now petered out.
Searching for a solution that didn't completely block <a href="https://www.torproject.org/">tor</a> access took longer than it could have.</p>
<!--
###VPS
So, I've got a VPS, and what do I use it for?
Well, IRC.
And as an IRC user it is always novel to have your own server/network.
-->
<h3 id="irc">IRC</h3>
<h4 id="ircd">IRCd</h4>
<p>I use <a href="https://www.unrealircd.org/">UnrealIRCd</a>. I like it, and it happens to be the flavor I first administered... back when that network still existed. Anyway, I like it, and I think it works well.</p>
<h4 id="irc services">IRC Services</h4>
<p>I realize it's taking me a while to get to the main topic, HOPM, but I might as well finish describing the setup at this point.
Anope is crap... And they happen to have some <a href="https://mikaela.info/english/2015/03/12/stay-away-from-anope.html">childish developers</a>...
When researching other IRC Services, <a href="https://atheme.net/">Atheme</a> was highly recommended in many places.
<strike>Unfortunately its development ceased.
But I did find two current forks, <a href="https://github.com/shalture/shalture">Shalture</a> being the one I chose.</strike>
Among the many things it does better, it has CertFP support.
<strong>NOTE 11DEC2016:</strong> Atheme is alive again, and Shalture's changes incorporated.</p>
<h5 id="certfp">CertFP</h5>
<p>CertFP is a nifty feature where, after teaching NickServ your SSL finger print, it automatically identifies you on connection. No commands needed. You do have to be connecting via SSL of course.</p>
<h3 id="botnet_2">Botnet</h3>
<p>Ok, now to the point.
A few weeks ago, some russian looking botnet decided to attack my VPS, and got on my IRCd.
Wasn't really doing much aside from having random seeming names, and making random channels... aside from spamming my console window with all the connections/exits.
But who wants to keeps someone else's botnet on their server?</p>
<h4 id="bopm">BOPM</h4>
<p>In my research to combat the botnet without completely blocking tor access, I again found a discontinued piece of software that <strong>everyone</strong> recommended... <a href="https://github.com/azzurra/bopm">BOPM</a>.
Unfortunately, not only has BOPM gone four years without and update, its documentation site has dropped of the face of the internet.
While I did take a shot at running BOPM, it did not want to compile for me...</p>
<h4 id="hopm">HOPM</h4>
<p>Well, after another few days of research, I dug up <a href="https://github.com/ircd-hybrid/hopm">HOPM</a>, an ongoing project that picks up the reins.</p>
<h5 id="what is hopm">What is HOPM</h5>
<blockquote>
<p>HOPM (Hybrid Open Proxy Monitor) is an open-proxy monitoring bot designed to
monitor an individual server (all servers on the network have to run their own
bot if the IRCD does not support `farconnect` user mode) with a local
operator {} block and monitor connections. When a client connects to a server,
HOPM will scan the connection for insecure proxies. Insecure proxies are
determined by attempting to connect the proxy back to another host (usually the
IRC server in question).</p>
<p>HOPM is written ground-up in C language and it is an improved fork of BOPM
(blitzed open proxy monitor), which is a concept derived from wgmon. It
improves on wgmon with HTTP support, faster scanning (it can scan clients
simultaneously), better layout (scalability) and DNSBL support.</p>
</blockquote>
<h5 id="use">Use</h5>
<p>It works very well, and configuration is pretty simple.
I'm going to skip the basic settings, as they are self explanatory.
Do note that there are a few <em>clearly</em> marked options that apply to specific IRCds.
I will briefly talk about enabling the black lists, as this is the meat of HOPM, and it took me a moment to grok it correctly.
Looking at the <a href="https://github.com/ircd-hybrid/hopm/blob/master/doc/reference.conf">sample conf</a> under the 'opm' section, you will see that there are four black lists to choose from, and settings for reporting bots if you choose.
I chose to use <a href="http://dronebl.org/">DroneBL</a> and <a href="http://rbl.efnetrbl.org/">EFnet RBL</a>, as DroneBL didn't catch everything on its own, and EFnet has survived for 25 years.</p>
<h5 id="conf">Conf</h5>
<div class="highlight"><pre><span></span><code><span class="c1">### OPM ### {{{</span>
/*
* OPM Block defines blacklists and information required to report new proxies
* to a dns blacklist. DNS-based blacklists store IP addresses in a DNS zone
* file. There are several blacklist that list IP addresses known to be open
* proxies or other forms of IRC abuse. By checking against these blacklists,
* HOPMs are able to ban known sources of abuse without completely scanning them.
*/
opm <span class="o">{</span>
/*
* Blacklist zones to check IPs against. If you would rather not
* trust a remotely managed blacklist, you could <span class="nb">set</span> up your own, or
* leave these commented out in which <span class="k">case</span> every user will be
* scanned. The use of at least one open proxy DNSBL is recommended
* however.
*
* Please check the policies of each blacklist you use to check you
* are comfortable with using them to block access to your server
* <span class="o">(</span>and that you are allowed to use them<span class="o">)</span>.
*/
/* dnsbl.dronebl.org - http://dronebl.org */
blacklist <span class="o">{</span>
/* The DNS name of the blacklist */
<span class="nv">name</span> <span class="o">=</span> <span class="s2">"dnsbl.dronebl.org"</span><span class="p">;</span>
/*
* There are only two values that are valid <span class="k">for</span> this
* <span class="s2">"A record bitmask"</span> and <span class="s2">"A record reply"</span>
* These options affect how the values specified to reply
* below will be interpreted, a bitmask is where the reply
* values are <span class="m">2</span>^n and more than one is added up, a reply is
* simply where the last octet of the IP address is that number.
* If you are not sure <span class="k">then</span> the values <span class="nb">set</span> <span class="k">for</span> dnsbl.dronebl.org
* will work without any changes.
*/
<span class="c1"># type = "A record reply";</span>
/*
* Kline types not listed in the reply list below.
*
* For DNSBLs that are not IRC specific and you just wish to kline
* certain types this can be enabled/disabled.
*/
<span class="c1"># ban_unknown = no;</span>
/*
* The actual values returned by the dnsbl.dronebl.org blacklist as
* documented at http://dronebl.org/docs/howtouse
*/
reply <span class="o">{</span>
<span class="nv">2</span> <span class="o">=</span> <span class="s2">"Sample"</span><span class="p">;</span>
<span class="nv">3</span> <span class="o">=</span> <span class="s2">"IRC Drone"</span><span class="p">;</span>
<span class="nv">5</span> <span class="o">=</span> <span class="s2">"Bottler"</span><span class="p">;</span>
<span class="nv">6</span> <span class="o">=</span> <span class="s2">"Unknown spambot or drone"</span><span class="p">;</span>
<span class="nv">7</span> <span class="o">=</span> <span class="s2">"DDOS Drone"</span><span class="p">;</span>
<span class="nv">8</span> <span class="o">=</span> <span class="s2">"SOCKS Proxy"</span><span class="p">;</span>
<span class="nv">9</span> <span class="o">=</span> <span class="s2">"HTTP Proxy"</span><span class="p">;</span>
<span class="nv">10</span> <span class="o">=</span> <span class="s2">"ProxyChain"</span><span class="p">;</span>
<span class="nv">13</span> <span class="o">=</span> <span class="s2">"Brute force attackers"</span><span class="p">;</span>
<span class="nv">14</span> <span class="o">=</span> <span class="s2">"Open Wingate Proxy"</span><span class="p">;</span>
<span class="nv">15</span> <span class="o">=</span> <span class="s2">"Compromised router / gateway"</span><span class="p">;</span>
<span class="nv">17</span> <span class="o">=</span> <span class="s2">"Automatically determined botnet IPs (experimental)"</span><span class="p">;</span>
<span class="nv">255</span> <span class="o">=</span> <span class="s2">"Unknown"</span><span class="p">;</span>
<span class="o">}</span><span class="p">;</span>
/*
* The kline message sent <span class="k">for</span> this specific blacklist, remember to put
* the removal method in this.
*/
<span class="nv">kline</span> <span class="o">=</span> <span class="s2">"KLINE *@%h 3d You have a host listed in the DroneBL. For more information, visit http://dronebl.org/lookup_branded?ip=%i&network=Network"</span><span class="p">;</span>
<span class="o">}</span><span class="p">;</span>
/* rbl.efnetrbl.org - http://rbl.efnetrbl.org/ */
blacklist <span class="o">{</span>
<span class="nv">name</span> <span class="o">=</span> <span class="s2">"rbl.efnetrbl.org"</span><span class="p">;</span>
<span class="nb">type</span> <span class="o">=</span> <span class="s2">"A record reply"</span><span class="p">;</span>
<span class="nv">ban_unknown</span> <span class="o">=</span> no<span class="p">;</span>
reply <span class="o">{</span>
<span class="nv">1</span> <span class="o">=</span> <span class="s2">"Open proxy"</span><span class="p">;</span>
<span class="nv">2</span> <span class="o">=</span> <span class="s2">"spamtrap666"</span><span class="p">;</span>
<span class="nv">3</span> <span class="o">=</span> <span class="s2">"spamtrap50"</span><span class="p">;</span>
<span class="nv">4</span> <span class="o">=</span> <span class="s2">"TOR"</span><span class="p">;</span>
<span class="nv">5</span> <span class="o">=</span> <span class="s2">"Drones / Flooding"</span><span class="p">;</span>
<span class="o">}</span><span class="p">;</span>
<span class="nv">kline</span> <span class="o">=</span> <span class="s2">"KLINE *@%h 3d Blacklisted proxy found. For more information, visit http://rbl.efnetrbl.org/?i=%i"</span><span class="p">;</span>
<span class="o">}</span><span class="p">;</span>
<span class="o">}</span><span class="p">;</span>
<span class="c1">### End OPM ### }}}</span>
</code></pre></div>
<p>So, it's not super complicated, uncommenting the parts you want to use and all.
It did take me a few minutes to realize the GLines were in the wrong format for UnrealIRCd though, and needed to be changed in three places (once for scan, and once for each enabled blacklist).</p>
<p>Instead of the default <code>kline = "KLINE 180 *@%h :Open proxy found on your host.";</code>, unreal wants <code>kline = "KLINE *@%h 3d Open proxy found on your host.";</code> (host then time, and no ':').</p>
<h3 id="conclusion_2">Conclusion</h3>
<p>Once you set HOPM up with your ban lists, it works very nicely.
Sure, your oper's get connect, kline, and exit spam; but now you have that warm, satisfied feeling from ban hammering.
In my limited use, it also does not appear to block legitimate <a href="https://www.torproject.org/">tor</a> users.</p>
<h3 id="links">Links</h3>
<p><a href="https://www.unrealircd.org/">UnrealIRCd</a><br/>
<a href="https://mikaela.info/english/2015/03/12/stay-away-from-anope.html">childish developers</a><br/>
<a href="https://atheme.net/">Atheme</a><br/>
<a href="https://github.com/shalture/shalture">Shalture</a><br/>
<a href="https://github.com/azzurra/bopm">BOPM</a><br/>
<a href="https://github.com/ircd-hybrid/hopm">HOPM</a><br/>
<a href="https://github.com/ircd-hybrid/hopm/blob/master/doc/reference.conf">sample conf</a><br/>
<a href="http://dronebl.org/">DroneBL</a><br/>
<a href="http://rbl.efnetrbl.org/">EFnet RBL</a><br/>
<a href="https://www.torproject.org/">tor</a> </p>pianobar, control-pianobar, and spectrwm2015-10-02T19:00:00-04:002015-10-02T19:00:00-04:00demuretag:demu.red,2015-10-02:/blog/2015/10/pianobar-control-pianobar-and-spectrwm/<p><strong>NOTE:</strong> I've since moved on to using i3wm, and this can be done nicely with i3 modes. You can see my i3 conf in my dotfiles.</p>
<h3 id="pianobar"><a href="http://6xq.net/pianobar/">pianobar</a></h3>
<p>So, <code>pianobar</code> is a nice command line pandora player.
Not much set up, besides telling it your username and passwd, hopefully using <code>password_comand …</code></p><p><strong>NOTE:</strong> I've since moved on to using i3wm, and this can be done nicely with i3 modes. You can see my i3 conf in my dotfiles.</p>
<h3 id="pianobar"><a href="http://6xq.net/pianobar/">pianobar</a></h3>
<p>So, <code>pianobar</code> is a nice command line pandora player.
Not much set up, besides telling it your username and passwd, hopefully using <code>password_comand</code>.</p>
<div class="highlight"><pre><span></span><code><span class="c1">#~/.config/pianobar/config</span>
<span class="nv">user</span> <span class="o">=</span> your_account
<span class="nv">password_command</span> <span class="o">=</span> pass your/pianobar
</code></pre></div>
<p>I'm using <code>pass</code> as the way I store passwds locally; it's nice enough.</p>
<h3 id="control-pianobar"><a href="https://malabarba.github.io/control-pianobar/">control-pianobar</a></h3>
<p>I ran into <code>control-pianobar</code> while considering how I could make pianobar play nice with conky. There aren't many ways...
After looking at control-pianobar, I liked the idea of binding pinaobar to system wide keys.</p>
<p>Installing control-pianobar is pretty simple.
Their site recommends the tar file, but I just used the git repo and linked the contents to my <code>~/.config/pianobar/</code>.</p>
<p>To start using it, you will have to add <code>event_command</code> with the full path to the script.</p>
<div class="highlight"><pre><span></span><code><span class="c1">#~/.config/pianobar/config</span>
<span class="nv">user</span> <span class="o">=</span> your_account
<span class="nv">password_command</span> <span class="o">=</span> pass your/pianobar
<span class="nv">event_command</span> <span class="o">=</span> /home/your_account/.config/pianobar/pianobar-notify.sh
</code></pre></div>
<p>As an aside, since I keep my config in my <a href="https://notabug.org/demure/dotfiles">dotfiles</a>, I set my <code>.gitignore</code> to:</p>
<div class="highlight"><pre><span></span><code><span class="c1">#~/.config/pianobar/config/.gitignore</span>
*
!.gitignore
!config
</code></pre></div>
<p>Otherwise a lot of cache files will be annoying:</p>
<div class="highlight"><pre><span></span><code>~ -> ls ~/.config/pianobar/
albumart downloadname pandora.jpg
artname durationstation pianobar-notify.sh
config durationstationexplain state
control-pianobar.sh isplaying stationlist
ctl log
downloaddir nowplaying
</code></pre></div>
<h3 id="spectrwm mapping"><a href="https://opensource.conformal.com/wiki/spectrwm">spectrwm</a> mapping</h3>
<p>So, for my window manager, I use <code>spectrwm</code>.
I like it since it reminds me a lot of <code>tmux</code></p>
<div class="highlight"><pre><span></span><code><span class="c1">#~/.spectrwmrc</span>
<span class="c1">### control-pianobar Commands ### {{{</span>
program<span class="o">[</span>piano-play<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh play
program<span class="o">[</span>piano-quit<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh quit
program<span class="o">[</span>piano-hist<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh <span class="nb">history</span>
program<span class="o">[</span>piano-next<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh next
program<span class="o">[</span>piano-tired<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh tired
program<span class="o">[</span>piano-ss<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh switchstaion
program<span class="o">[</span>piano-ps<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh previousstation
program<span class="o">[</span>piano-ns<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh nextstation
program<span class="o">[</span>piano-love<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh love
program<span class="o">[</span>piano-ban<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh ban
program<span class="o">[</span>piano-expl<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh explain
program<span class="o">[</span>piano-curr<span class="o">]</span> <span class="o">=</span> ~/.config/pianobar/control-pianobar.sh current
<span class="c1">### End control-pianobar Commandss ### }}}</span>
<span class="c1">### control-pianobar Keys ### {{{</span>
bind<span class="o">[</span>piano-play<span class="o">]</span> <span class="o">=</span> MOD+F12
bind<span class="o">[</span>piano-quit<span class="o">]</span> <span class="o">=</span> MOD+F11
<span class="c1">#bind[piano-hist] = MOD+F</span>
bind<span class="o">[</span>piano-next<span class="o">]</span> <span class="o">=</span> MOD+F1
bind<span class="o">[</span>piano-tired<span class="o">]</span> <span class="o">=</span> MOD+F2
bind<span class="o">[</span>piano-ss<span class="o">]</span> <span class="o">=</span> MOD+F7
<span class="c1">#bind[piano-ps] = MOD+F</span>
<span class="c1">#bind[piano-ns] = MOD+F</span>
bind<span class="o">[</span>piano-love<span class="o">]</span> <span class="o">=</span> MOD+F3
bind<span class="o">[</span>piano-ban<span class="o">]</span> <span class="o">=</span> MOD+F5
<span class="c1">#bind[piano-expl] = MOD+F</span>
bind<span class="o">[</span>piano-curr<span class="o">]</span> <span class="o">=</span> MOD+F8
<span class="c1">### End control-pianobar Keys ### }}}</span>
</code></pre></div>
<h3 id="links">Links</h3>
<p><a href="http://6xq.net/pianobar/">pianobar</a><br/>
<a href="http://www.passwordstore.org/">pass</a><br/>
<a href="https://malabarba.github.io/control-pianobar/">control-pianobar</a><br/>
<a href="https://notabug.org/demure/dotfiles">dotfiles</a><br/>
<a href="https://opensource.conformal.com/wiki/spectrwm">spectrwm</a><br/>
<a href="https://tmux.github.io/">tmux</a> </p>This is a Test2015-09-18T17:08:00-04:002015-09-22T12:26:00-04:00demuretag:demu.red,2015-09-18:/blog/2015/09/this-is-a-test/<p>This is a test<br/>
blah blah blah </p>
<p>Bacon ipsum dolor amet ground round short loin ribeye bacon prosciutto. Leberkas venison andouille, chuck fatback flank spare ribs chicken tenderloin pork chop doner filet mignon ham hock shoulder beef ribs. Strip steak kevin beef spare ribs. Shoulder salami strip steak, short loin …</p><p>This is a test<br/>
blah blah blah </p>
<p>Bacon ipsum dolor amet ground round short loin ribeye bacon prosciutto. Leberkas venison andouille, chuck fatback flank spare ribs chicken tenderloin pork chop doner filet mignon ham hock shoulder beef ribs. Strip steak kevin beef spare ribs. Shoulder salami strip steak, short loin bacon andouille rump short ribs chicken sirloin. Turkey ham hock flank chuck, frankfurter cow meatball leberkas corned beef ground round. Ribeye flank cupim, shankle meatball beef ribs pork loin. Landjaeger pig ball tip pork chop prosciutto pork tail chuck venison flank filet mignon pastrami ham pancetta cupim.</p>
<p>Shoulder sausage brisket salami. Tri-tip kevin beef ribs pork loin shank drumstick shoulder leberkas chicken frankfurter spare ribs short loin bacon. Short ribs pork kevin, meatball venison andouille frankfurter ground round drumstick turducken ribeye biltong pig sirloin shankle. Pork loin corned beef landjaeger flank, frankfurter bresaola prosciutto cupim ham meatloaf. Kielbasa biltong leberkas, prosciutto bacon doner meatloaf sausage t-bone pastrami.</p>