AWS-S3 complaining at you in Ruby 2.1.1?
Looks like they’re getting rid of OpenSSL::Digest::Digest and that behavior should be in OpenSSL::Digest instead.
Nix it here: aws-s3-0.6.3/lib/aws/s3/authentication.rb:71.
If I have a chance I’ll submit a PR on that (or maybe it’s fixed in a newer version).
Edit: The change is already present in the master copy of aws-s3 here
Missing your db:migrate tasks in Rails?
Might have been disabled during initial app work. Check config/application.rb and see if require "active_record/railtie" is commented out.
Version: Rails 4.x.
My use case is pushing code that resides on Github as well as on Bitbucket. I want it available in both remote locations in case one is unavailable.
Here’s how you do it:
Add the two remotes as normal
git remote add origin GIT_LINK_TO_REPO
git remote add bitbucket GIT_LINK_TO_REPO
In local repo, edit .git/config. Find the entry for origin and bitbucket
[remote "origin"]
url = git@github.com:zph/zph.git
fetch = +refs/heads/*:refs/remotes/origin/*
[remote "bitbucket"]
url = ssh://git@bitbucket.org/zph/zph.git
fetch = +refs/heads/*:refs/remotes/bitbucket/*
Add a new entry using that information and urls:
[remote "all"]
url = git@github.com:zph/zph.git
url = ssh://git@bitbucket.org/zph/zph.git
Now when pushing code:
git push all
Credit for this solution: http://stackoverflow.com/questions/849308/pull-push-from-multiple-remote-locations?lq=1
You’re working for Cato Consulting.
And, you’ve been lent out to a new client on an unfamiliar project. It’s a codebase that you haven’t touched before. In the sense that you’ve never worked on it, you could kind of call it “Greenfield”… but from where you sit, the project looks like “Brownfield with withered vestiges of healthy code.” This isn’t going to be an easy upcoming two weeks.
So it’s a new day and you have a new story from the client about a text input box that’s no longer working. It should behave as a search field, but instead it does nothing. You realize you also do nothing useful before coffee while sitting there, pondering the solution and decide to increase your caffeine level.
Much better, there can be coding once the caffeine hits.
First, you need the text field. Perhaps it will have a unique id or class.
Bingo, you learn that the field has an id of “#super_search_box”.
Your other big clue in the story is that this feature was working well up until a few weeks ago. The client doesn’t have a schedule of exactly when it went bad, but that gives you a starting point. You can reasonably estimate that the code from 4 weeks ago was working.
Let’s dig into this Git Archaeology! We’ll dirty our hands but our spirits will be clean and we’ll sleep well at night knowing that our problem domains were well understood when implementing a fix.
Our first useful incantation is git bisect. Git bisect helps in a situation where something stops working.
The basic workflow looks like this:
(Starting on git branch = master at newest commit, which happens to be broken)
git bisect start
(Then identify a good and a bad commit.)
git checkout SHA_from_4_weeks_ago
(manually check that search box worked, or run associated automatic tests)
If it’s good, mark it via git bisect good, otherwise git bisect bad
Then find an opposite example (ie if you found a bad one, find a good working commit).
Tag that one via a git commit [good || bad]
Then git bisect’s magical excavation will begin.
Git will do a binary search, and each step of the way, you will enter git bisect [bad || good] until git bisect identifies the earliest commit that broke that feature.
After that process is over, you’ll have the bad commit where a breaking change was introduced to that feature.
Next, it’s time to understand why the code author would do such a dastardly thing!
Run a git show to see the full content of that commit, both the message and the code differences.
(Note that you can see which were marked as good/bad by checking out git log --oneline)
If there are more than 2 people on the project, hopefully there’s a central project management tool like Pivotal Tracker or Jira. Ever hopeful, you expect to see a story number or issue number listed in the git commit. Let’s see what it looks like:
[CTL] [#3443450] Switched JQuery selector for search box
Changes selector from '#super_search_box' to '.snazzy_search'.
diff monolithic_everything.coffee
- '#super_search_box'
+ '.snazzy_search'
Not a very helpful message but at least it contains a story id. Looking that up in the Project Tracker show that this was a chore to refactor some code & apparently it wasn’t done with sufficient care or any automated behavioral tests. Bummer :(.
Since we’re left underwhelmed but the currently available info, it would be nice to know when the markdown was modified that relates to ‘.snazzy_search’
git log -S 'snazzy_search'
This will search each git log entry for any mentions of snazzy_search.
With knowing when the markdown was changed, along with knowing the exact commit that introduced the regression, you’re set to implement a fix. By doing a bit of digging, the underlying domain becomes clear & the possibility of introducing your own regression will decrease.
Check out my .gitconfig and my .zsh.d/git.zsh files here for some helpful shortcuts for everyday git behaviors.
Also, look into git blame via Fugitive.vim or Magit (on Emacs). Git blame’s a great way to find out if the developer who introduced the changes is still around and can give you a bit of context on the changes.
If you’re just getting started as a Git Archeologist and want some help with getting started, or if you’re an experienced Git Excavator and want to bounce ideas off me, ping me on Twitter at @_ZPH.
For further reading, checkout these two great blog posts: http://ruturaj.net/git-bisect-tutorial/ and http://mislav.uniqpath.com/2014/02/hidden-documentation/
Want to know the secret about tech conferences? It’s not about the talks. It’s not about the venue.
It’s about all the people who are there!
While attending @RbOnAles, I was asked by @eliserius about how I knew so many people there. This article is a summary of that discussion.
First off, go to a couple conferences each year. It’s best if your employer will pay all or part of the way. But if they won’t, it’s your responsibility to make it to the conferences. They’re an investment in your career’s future.
Now that you’re attending 3+ regional tech conferences each year, what steps can you take to make the most of it? Well, remember that you’re attending to meet the other attendees. By virtue of them also attending the conference, they either work at decent companies or they prioritize conference attendance.
So meet these folks! It’s not hard*. (It’s terrifying at first, but push past that & make it happen). I’m an introverted person and know a fair number of Ruby conference attendees, but those first few minutes of awkwardly interacting are just that: uncomfortable. That’s fine, move the conversation into tech stuff. Chat about Service Oriented Architecture, argue about Dependency Injection, take a stand and say that Ruby isn’t a dying language.
Remember, these folks who are at the conference also want to meet people.
What specific steps do I recommend for breaking the ice or stacking your social deck at the conference?
$EDITOR to use. If you’re stumped about approaching someone, just be candid, “I saw you standing there & realized we hadn’t met yet: I’m Zander”. Most of the time that’s enough to kick off a conversation, though you might choose to use your own name in the prior quotation.Also, I’m terrified of starting conversations… so be my guest and come up & tell me you read this post. That’ll break the ice!
PS - I have a new friend from @RbOnAles who’s looking for a remote role (or in SF) either as QA or Jr. Ruby Developer. DM me on Twitter if you know of options.
Over the weekend I finally admitted to myself that I hate submodules.
But they’re a keystone to one of my primary development tools: Vim. In order to use Vim, I use the Pathogen.vim plugin by @tpope. In order to use Pathogen, you normally use submodules in the .vim/bundle/ folder.
But submodules are the work of the devil.
I tried out the Vundle plugin and was seeing much longer loadtime for vim:
So I asked around on Twitter and @jwieringa advised me to checkout ‘infect’ by @crsexton .
And infect is awesome! It works with Pathogen to give it a declarative style for plugins. An example is:
"=bundle tpope/vim-pathogen
"=bundle tpope/vim-sensible
source ~/.vim/bundle/vim-pathogen/autoload/pathogen.vim
execute pathogen#incubate()
"=bundle mileszs/ack.vim
"=bundle vim-scripts/AutoTag
"=bundle kien/ctrlp.vim
"=bundle Raimondi/delimitMate
"=bundle sethbc/fuzzyfinder_textmate
"=bundle tpope/gem-ctags
"=bundle gregsexton/gitv
"=bundle sjl/gundo.vim
"=bundle tpope/vim-vinegar
"=bundle jnwhiteh/vim-golang
"=bundle wting/rust.vim
call pathogen#helptags()
set nocompatible " We're running Vim, not Vi!
syntax on " Enable syntax highlighting
filetype on " Enable filetype detection
filetype indent on " Enable filetype-specific indenting
filetype plugin on " Enable filetype-specific plugins
Use it. Love it. Don’t look back!
If you want faster downloads with ‘infect’ try this unofficial fork of the standalone: https://github.com/zph/zph/blob/master/home/bin/infect. When I have time, I’ll work w/ @crsexton to get this added to ‘infect’.
Blame smartOS not reading my SATA controller.
Blame Linux for not having first class support for ZFS (where my data is held).
Blame @canadiancreed for giving me a way out of the quandry.
The backstory is that I moved all of my backups to two ZFS pools a few years ago. I was running ZFSonLinux and it generally worked…. except when I rebooted the server and had to force import the pools.
Fast forward to that server being replaced by a workstation (i7 Haswell, 240GB Sata3 SSD, and 24GB RAM). I tried smartOS by Joyent and ran into issues with my SATA controller not being recognized. It’s a shame given how awesome the smartOS vm administration is. I mean vmadm and imgadm are light years ahead of Docker.
Since smartOS refused to recognize 3 of my 8 drives I ran back to Linux. The good new, Linux recognized the sata controller. The bad news, Linux couldn’t import the pools. Frankly, Linux had a hell of a time with building the ZFS on Linux kernel modules. I managed to piece together a few clues from the ZFSonLinux Issues on Github. In fact, thanks to @dajhorn for supplying answers on that Github issue which allowed me to build the kernel modules.
This sounds promising doesn’t it? It’s not, it was a horrorshow. Linux refused to import the one of the two zpools. One of them imported nicely, the mission critical financial records. The other semi-critical zpool was shot and refused to import under Linux.
I spent a whole evening banging my head against this issue. If it hadn’t been for Chris Reed I wouldn’t have hit upon the solution.
His recommendation was to try FreeBSD… so I did. And it recognized the SATA controller and also imported the zpools cleanly! So after a dry run w/ FreeBSD, I installed PC-BSD, which is a desktop variant based on FreeBSD. Think of it as the Ubuntu of the BSD world. And hell yeah, it’s all working :).
So far, I’m really liking it. Replace ‘aptitude’ with ‘pkg’ and it’s pretty similar. Except, PC-BSD is working where Linux & ZFS were a hassle.
RVM installation went poorly on FreeBSD.
The ca-certificates weren’t up to date according to the install script.
Really, the ca-certificates weren’t in the right location for RVM’s curl install script.
These commands as root fixed it:
mkdir -p /usr/local/opt/curl-ca-bundle
ln -s /usr/local/share/certs/ca-root-nss.crt /usr/local/opt/curl-ca-bundle/share/ca-bundle.crt
I ran into an issue with using xargs with rm -rf.
This could be dicey, so get your safety hats on.
The issue was that the filename included an apostrophe. So when trying to do a simple command such as tree -fi | grep conflicted | xargs rm -f '{}', I received an error about having an unterminated quote in the parameter.
Apparently, some versions of xargs allow you to specify a delimiter with a -d but the copy on my Mac didn’t have such a flag.
Instead, I learned that egrep --null will use a null character as the divider between matches. So the following solved my problem:
tree -fi | egrep --null conflicted | xargs -0 rm -f '{}'
Let’s break that command set down:
tree -fi is a trick I recently learned from my friend @olleolleolle. It prints out the local tree of files and the -fi prints out the whole filename (including directories).egrep --null conflicted is where the magic starts. The --null flag tell egrep to separate matches with a null character.xargs -0 rm -f '{}' this tells xargs that the null character is divider and to remove each filename that comes through the linux pipe.This is yet another example of why I’m impressed with the commandline. Simple little tools that can be chained together with symbiotic behavior.
Last Friday was the kind of day where I dropped into a Pry repl in order to bang out an automation script.
The challenge was: automate the retrival of specific emails that contained receipts, wrangle them into sane data structures, and dump them into a spreadsheet with both daily totals and an absolute total.
An example email looked like this:
Receipt #9999999999999
County:Example, FL Date: 2014-1-27
Name:Jill Doe
Credit Card #XXXXXXXXXXXX9999
Authorization code:999999
No.of Pages viewed:3
Total Amount: $ 3.00
Thank you for visiting http://www.example.com
First step was to build an email parser for this format. I tried to keep it tolerant of future changes to the email generation scheme.
The general steps involved are:
The full code for that module is listed below.
module Email
class Parser
attr_accessor :email, :content
def initialize(email)
@raw_content = email.to_s
@content = @raw_content.split("\n").map(&:strip)
end
def receipt
content.grep(/receipt/i).first[/\d+/]
end
def county_line
@county_line ||= content.grep(/county.*date/i)[0]
.split(/\W{3,}/)
.map { |i| Hash[*i.split(':').map(&:strip)] }
end
def county
county_line.first["County"]
end
def date
county_line[1]["Date"]
end
def name
array = content.grep(/name/i).first.split(':')[1]
end
def credit_card_number
content.grep(/credit card/i).first[/#.*$/]
end
def authorization_code
content.grep(/authorization code/i).first.split(':')[1]
end
def pages_viewed
begin
content.grep(/pages viewed/i).first[/\d+/].strip
rescue NoMethodError => e
warn "#{e.message} for #{content.inspect}"
end
end
def total_amount
content.grep(/total amount/i).first
.split(':')[1]
.gsub(/\$/, '')
.strip
end
def website
content.grep(/visiting http/i).first[/http.*$/i]
end
def all
ParsingPresenter.new(
county: county,
date: date,
name: name,
credit_card_number: credit_card_number,
authorization_code: authorization_code,
pages_viewed: pages_viewed,
total_amount: total_amount,
website: website,
receipt: receipt
)
end
def self.all(msg)
ps = new(msg)
ps.all
end
end
ParsingPresenter = Class.new(OpenStruct)
With that in order, I set about using the awesome ruby-gmail gem for retrieving said emails. Note: after completing this project, I learned of a continuation of the ruby-gmail gem called gmail. All the code in these examples is specific to the older incarnation of the gem.
ruby-gmail has a simple interface for retrieving messages between date ranges. So I setup a specific Gmail filter for emails from a certain sender that included the text ‘receipt’.
There’s nothing too fancy in this code, but it’s important to set @gmail.peek = true so that programatically viewed emails aren’t marked ‘read’. Also of note is the use of Dotenv for setting secret values without risking them in a git repo.
class Retriever
USER = ENV['GMAIL_USER']
PASSWORD = ENV['GMAIL_PASSWORD']
LABEL = 'Receipts'
attr_accessor :user, :password, :gmail, :messages
def initialize(user=USER, password=PASSWORD)
@user = user
@password = password
@gmail = Gmail.new(@user, @password)
@gmail.peek = true
end
def message_count_in_range(start_date, end_date)
#dates as '2010-02-10'
gmail.inbox
.count(:after => start_date, :before => end_date)
end
def emails_in_range(start_date, end_date, label=LABEL)
#dates as '2010-02-10'
gmail.mailbox(label)
.emails(:after => start_date, :before => end_date)
end
def message_presenters_in_range(start_date, end_date)
msgs = emails_in_range(start_date, end_date)
@messages = msgs.map do |msg|
Presenter.present(msg)
end
end
end
class Presenter
attr_accessor :email
def initialize(msg)
@email = msg
end
def body
email.body
end
def date
email.date.to_date
end
def date_string
date.to_s
end
def self.present(msg)
presenter = new(msg)
Message.new(date: presenter.date_string, body: presenter.body)
end
end
Message = Class.new(OpenStruct)
The last task in building this tool was dumping the data to a CSV with totals by date as well as a grand total. The process is simple, pass in a collection of messages and iterate through them by date, add a subtotal per date, then add a final row with grand total.
I like to break out rows into their own methods when possible. In fact, were I to rewrite this code, the message row would have its own method to clean up the inner loop of messages_by_date(). Another trick that helped for testing was to not generate a file on the filesystem. CSV takes either an open or a generate method. With generate it will pass the complete csv file out as the return value!
class CSVBuilder
attr_accessor :messages, :csv
def initialize(messages)
@messages = messages
end
def create
@csv = CSV.generate do |csv|
csv << header
csv << empty_row
uniq_dates.each do |date|
messages_by_date(date).each do |msg|
csv << [msg.date, msg.receipt, msg.authorization_code, msg.pages_viewed, msg.name, msg.credit_card_number, msg.total_amount]
end
csv << sum_totals_row(messages_by_date(date), "Subtotal for #{date}")
end
csv << empty_row
csv << sum_totals_row(messages, "Total Amount")
end
end
def header
['Date',
'Receipt #',
'Authorization Code',
'Pages Viewed',
'Name',
'Credit Card #',
'Total Amount']
end
def empty_row
Array.new(header.count)
end
def messages_by_date(date)
messages.select { |m| m.date == date }
end
def uniq_dates
messages.map(&:date).uniq.sort
end
def sum_totals_row(msgs, label)
rawsum = msgs.map { |m| m.total_amount.to_f }.inject(:+)
sum = sprintf( "%.2f", rawsum )
sum_row_padding = Array.new(header.count)
sum_row = [ sum_row_padding, label, sum ].flatten
end
end
I’m also quite proud of the variable name rawsum because it’s rawsome to design code that will save a couple hours every two weeks.
With a good ecosystem of libraries, it’s only a couple hours of work to write a re-usable tool that saves significant amounts of time. Hooray :).