Pundit and Rolify Testing with Rails

Just ran into an oddity that I didn’t expect (due to a false assumptions) when testing permissions and wanted to send it out to the world.

I have an app with two types of roles; :admin or :user. I am using Rolify in combination with Pundit to perform the role based authorization of REST actions.

I mixin a module containing general purpose authorization classes. I do it this way because permissions schemes tend to change alot and centralization makes it easier to change. You could easily just add these to your pundit policy classes.

def is_admin?
  user.has_role?(:admin)
end

def is_user?
  user.has_role?(:user)
end

def is_allowed?
  user.has_role?(:user, record)
end

Pretty straight-forward. An administrator has the role of :admin, a user has a role of :user and a :user is only allowed to access records that they are explicitly granted permission on.

My error was the assumption that user.has_role?(:user) would return true if a user had a role of :user explicitly set on any object in the system. In a pundit action it would look something like this.

def index?
  is_admin? || is_user?
end

The problem is that this was returning false unless someone had a :user role set in general, not on a specific instance of a record.

I think code clears this up further, here is the error.

#Assign the user role of :user on a specific object instance
User.first.add_role(:user, SomeObject.first)
User.first.has_role?(:user) #= false

#Assign the user a role of :user
User.first.add_role(:user)
User.first.has_role?(:user) #= true

So i had to alter this to use see if the User had the role of :user on any object in the system.

#Old definition, broken
def is_user?
  user.has_role?(:user)
end

#New definition, fixed!
def is_user?
  SomeObject.find_roles(:user, user).any?
end

The new is_user? returns true when the user has a role of :user on any SomeObject in the system. Which is exactly what we need.

Lesson learned, I assumed that the method worked a certain way and lost several hours of debugging in my unit tests. I normally will go straight to IRB when i run into anomalies but because i strongly help my assumption I didn’t.

Rails 4.2 truncate_words method

Rails 4.2 added a new feature called truncate words. It allows you to do things like this.

"In a world where everything is awesome".truncate_words(3)
"In a world..."

I wanted to step through this method to look at a few things I found in the code. Here is the full code

def truncate_words(words_count, options = {})
  sep = options[:separator] || /\s+/
  sep = Regexp.escape(sep.to_s) unless Regexp === sep
  if self =~ /\A((?:.+?#{sep}){#{words_count - 1}}.+?)#{sep}.*/m
    $1 + (options[:omission] || '...')
  else
    dup
  end
end

The first thing that I noticed is that it recognizes two additional options
:separator allows you to change the thing that splits up words. For instance, you could do something like cut off the string at the third i. That would look like this.

"Once i went to india and ate ice".truncate_words(3, {separator: 'i'})
"Once i went to india and ate ..."

There is also :omission, which allows you to add something besides … at the end of the truncation.

Now that we know the options, lets look at the code.

The first two lines setup the separator that is used to count the number of words. The first line either takes it from the options hash passed in or defaults it to a space. The second line converts the separator into regular expression language by escaping it (i.e. adding 1 or more \) UNLESS the separator is already of class Regexp. I suspect the unless is so that we don’t get double escaping that isn’t handled already in the Regexp library.

sep = options[:separator] || /\s+/
sep = Regexp.escape(sep.to_s) unless Regexp === sep

Now that we have our separator Regex’ed out, we check the string for it’s match. Self is going to be some sort of string that responds to matching a regex. If it finds a match the truncate_words will return the match (which is stored in the $1 global variable) along with either the specified omission or ‘…’ at the end. If we don’t get enough matches, then the string is just duplicated out.

if self =~ /\A((?:.+?#{sep}){#{words_count - 1}}.+?)#{sep}.*/m
  $1 + (options[:omission] || '...')
else
  dup
end

Something I’m wondering about is why do they duplicate the string at the end if no match is found? Wouldn’t just sending out self work and not eat up a tiny bit more memory?

In my next post I analyze the regex that is used to do matching on this string.

Thoughts on my first EmberJS project

Just cooked up an EmberJS project and wanted to share some thoughts on the framework as a whole and also throw some gotchas.

1. Many things work out of the box. However, when you need to do something that is not out of the box, you will find few resources to help. Worse yet, things change so much that found resources may not.

2. Some things are shockingly easy to do. Some of those things include data validations, 2 way binding and active record (rails) integration with active model serializer.

3. Some things are shockingly hard to do. For instance, looking over a javascript object is pretty simple in javascript or jQuery. However, in ember (with handlebars) you have to convert it into an ARRAY first, then it will loop pretty easily. This surprised me because EmberJS is javascript… looping over and displaying JSON should be pretty easy.

If I had one recommendation, it is to not use templating languages or extensions like emblem. I used this during my first project. It is similar to slim or haml and makes it super easy to see how the markup will be. But the downside is that whenever you need to do something unique, you will fight with it in order to get it implemented.

Javascript frameworks look like they will be the way of the future so good luck if you get started!

Adding Nested Image Attributes to Rails 4 App using Paperclip

So have a new web application where users will be able to upload images along with a post. I haven’t used the ruby gem Paperclip from ThoughtBot since rails 3.2 (which seems like it was only a couple months ago) so I thought I would write up a post on how I got it working.

The background of the app

  • Post have many images
  • Image is polymophic and can belong to either Post or Comment

So first step is to add the gem ‘paperclip’ to your gem file and then run bundle install.

Next, configure paperclip in development/production.rb   For this app we will be doing file storage locally until it is live

config/enviornments/development.rb

Paperclip.options[:command_path] = "/usr/local/bin"

After that we need to setup our models for Image and Post. Image is polymorphically related to Post as imageable and also contains content through the paperclip gem. Post has images and can accept nested attributes for images.

app/models/image.rb

class Image < ActiveRecord::Base
  belongs_to :imageable, :polymorphic => true
  has_attached_file :content, :styles=>{:medium => "300x300>", :thumb => "100x100>"}
  validates_attachment_content_type :content, :content_type => %w(image/jpeg image/jpg image/png)
end

app/models/post.rb

class Post < ActiveRecord::Base
  belongs_to :user
  has_many :images, :as => :imageable, dependent: :destroy
  accepts_nested_attributes_for :images
end

Now that the models are good, we have to code around the strong parameters that were added in rails 4. Notice that in the private section we are adding ‘images_attributes: [:content]‘ to prevent the image upload from being filtered and adding @post.images.build to the new action so that our view can render out the file tag properly.

app/controllers/posts_controller.rb

class PostsController < ApplicationController
  
  def new
    @post = Post.new
    @post.images.build
  end  

  def create
    @post = Post.new(post_params)
    @post.user_id = current_user.id

    respond_to do |format|
      if @post.save
        format.html { redirect_to @post, notice: 'Post was successfully created.' }
        format.json { render action: 'show', status: :created, location: @post }
      else
        format.html { render action: 'new' }
        format.json { render json: @post.errors, status: :unprocessable_entity }
      end
    end
  end

  private
    #strong parameters
    def post_params
      params.require(:post).permit(:title, :description, images_attributes: [:content])
    end
end

Last part is the view, now that it is wired up it will just be adding a couple lines to our form. I removed the code that will spit out the errors that come through for succinctness here. Note that the upload image part has :images in fields_for which corresponds to our has_many :images in Post model and that the file_file :content corresponds to the has_attached_file :content in our Image mode.

Also, in order for the html field to accept images, make sure you add the multipart to the html.

app/views/posts/_form.html.erb

<%= form_for @post, :html => { :multipart => true } do |f| %>
  <div class="field">
    <%= f.label :title %><br>
    <%= f.text_field :title %>
  </div>
  <div class="field">
    <%= f.label :description %><br>
    <%= f.text_field :description %>
  </div>
  <div>
    Upload Image
    <%= f.fields_for :images do |ph| %>
      <%= ph.file_field :content %>
    <% end%>
  </div>

  <div class="actions">
    <%= f.submit %>
  </div>
<% end %>

That about does it. If you put this together and it doesn’t work, restart your server and it should start singing.

Exporting Data to CSV in Ruby on Rails

So in our application we have a list of clients that our users are tracking (called trackers). One of our clients needed the ability to quickly export this data. In ruby this is super easy. We just add a few lines to our controller and model. In our controller, we already have an instance of their active trackers (@trackers_enabled )and it just grabs the trackers that are enabled.

trackers_controller.rb

respond_to do |format|
  format.html
  format.csv { send_data @trackers_enabled.to_csv}
end

This allows us to export the data by just appending .csv to the url, so to access this we just add a link in the view. In the model we just add.

tracker.rb

  
def self.to_csv(options = {})
  CSV.generate(options) do |csv|
    csv << column_names
    all.each do |row|
      csv << row.attributes.values_at(*column_names)
    end
  end
end

This is a quick solution and can be found in about 20 places around the internet, but it adds some unnecessary fields like created_at, updated_at and user_id to the csv.

To limit that data we just have to do some investigation on the column_names and see what type of data it returns in IRB.

In irb

Tracker.column_names
=>  ["id", "user_id", "address", "first_name", "last_name", "enabled", "created_at", "updated_at"]

So now we know that column names is just an array of the columns. To grab just the ones we are interested in we edit tracker.to_csv to have the columns we want or we could edit column_names to just specify the columns we want. I opted with the first because it won’t have any unintended consequences.

tracker.rb

  
def self.to_csv(options = {})
  CSV.generate(options) do |csv|
    columns = %w(first_name last_name age address description)
    csv << columns
    all.each do |row|
      csv << row.attributes.values_at(*columns)
    end
  end
end

Then in the view code you can add something like this if you are using bootstrap.

<div class="container-fluid padded">
  <%= link_to "Export #{@trackers_enabled.count} Trackers to CSV", params.merge(:format => :csv), :class => "btn btn-green" %>
</div>

Lockerware Part 2: Combatting the Threat Using Email Best Practices

In my last post I explained how lockerware was a new type of virus that encrypts a users data and gives them an option to pay a ransom to get it back. This post will be dedicated to the three options that this new persistent threat presents: ignore it, cure it or pro-actively avoid it.

I will combine ignoring and curing together because they are related and simple. Ignoring the threat can be done by just doing business as usual; continue to download all or most attachments from email, click on internet and email links without scrutiny and install random software from the internet. Following this route will surely get some form of Lockerware on your system. Once infected you can either just erase all your data and start fresh or pay several hundred or thousand dollars for the criminals to unlock your data so that you can use it.

A pro-active strategy is probably where most people want to be, however it will involve sacrificing some convenience for the extra security.

First, have a solid backup strategy in place. This strategy will be different for consumers vs businesses, however the common element is to have 3 copies of your data. The first is the data itself. The second is a backup copy that is stored onsite on a separate drive, preferably in a separate area of the building. The third is a backup copy that is stored far enough away that a local/regional disaster would not whipe out all data.

Second, be critical of the things you receive in email. Email is similar to a postcards because the sender could easily forge where it is coming from. So in email, it is trivial for a hacker to set their email as being from “Bank of America”. This trick is particularly bad because users have a tenancy to grant trust to email from respected sources like Bank of America or the FBI that they would not to other sources. This trust usually causes users to ignore the common preventative steps that I recommend at the end of this article.

Hackers also use email content to trick people. The most common method is through clickable links. People get links all the time that may say something like “watch this super cute cat video” but the truth is that the link that is sent can be programmed to say one thing but send users who clicks it to a completely different site. In English, that means that if you click cat video link, it could actually send you to a malicious website. Even worse that site could contain a virus that your phone or computer could download and run automatically.

A great real world example of these email tricks is from a friend that is paid to test banking security. During his last test he was able to hack a Bank President through email links. He was able to do this by researching the President on facebook. He discovered the the President was a member of a quilting club. Using that knowledge, he crafted an email that appeared to be from the quilting organization that had a link to a new membership information file. By clicking the seemingly trusted link, the President unintentionally put her entire organization at risk. Luckily though, it was just a drill and an opportunity to improve security.

So how do you avoid being fooled through email? Here are the guidelines that I recommend.

1. If you did not ask for or expect the file, don’t open it, ever
2. Don’t believe the return address…just because it says it is from your bank or friend does not mean that is who actually sent it
3. Even if the email is from your friend, assume that the contents are malicious and scrutinize the content
4. If you have to visit an email link, copy it to your clipboard and past it into your browser
5. If you have to download something from email, scan it with you anti-virus before opening it

By following the steps above you will avoid a good portion of the bad things that are sent through email.

Lockerware: The New Era of Viruses and Ransomware

Over the past several years we have saw the evolution of bad software that can get on people’s computers.  When these things were first created they were mostly malignant and the result of curious people that were just wanting to poke around at the internet.

Over time this type of activity has evolved into crime syndicates using software that will spy on users, send spam email, collect banking credentials and grant bad guys access to computers so that they can use it hack other computers.   The more techsavvy of you will recognize these systems as Botnets, Zombie Farms, Zues Banking Trojans and Spambots.

However, we recently experienced a game changer.   A new set of software is making criminal upwards of millions of dollars per month in a completely untraceable way.   This software is called Lockerware.  The pioneer of Lockerware is called Cryptolocker and was first detected in early 2013.  It works by using military grade security to scramble the files, documents, pictures and other data on a computer using a key, this is called encryption.  It then sends the key off of your computer and will only release the key to undo the damage if a ransom of 300-2000 so dollars is paid using untraceable virtual currency.   Users know they have it when they get a popup that says “Your files are encrypted, pay us $1000 in 96 hours or we will delete the key.”  A timer then begins to count down to zero.  If you pay the random they give you the key and you get your documents back, otherwise they are gone forever unless you had a solid backup in place.

These guys have made a TON of money.  An estimated 3% of users pay the ransom and around 12,000 new infections are reported per week ( that would amount to 360,000 dollars at 1000 per paid infection).  At hundreds of thousands of dollars in profit per week, it is probably not a surprise that a half dozen variations of Lockerware have been reported.  What you wouldn’t expect is that a 23 year old developer has recently released a $100 toolkit that people can buy that includes the code to create Lockerware.   Furthermore, the next logical step in the evolution of this type of software will be for the software to be smart enough to spread on its own across the internet.  This is the equivalent of some aweful virus like ebola becoming air-borne and WHEN it happens there will be alot of people and companies that lose access to their data.

To prepare for this upcoming evolution in virus software people have a 3 choices going forward.  They can Ignore & Cure it or Prevent it.

In my next article I will address steps that the every day computer user can take to prevent Lockerware from getting on their computers.

Pattern Lock Passwords…Not as many choices as you would think

Phones currently give the option of using a 9 dot pattern to unlock the phone. Although this is not perfect, it is better than nothing. However, I was thinking the other day, most people use the same password, I wonder how many use the same pattern. Furthermore, how many possible patterns are there given a number of dots? I will be addressing the later question in this post and will collect input from readers to determine the former.

The pattern I will be using is the common 3 rows of 3 dots. I will number the dots from left to right. The top will be 1, 2, 3. The middle row 4, 5, 6 and bottom 7, 8, 9. I will also be calling a single continuous swipe amongst many numbers a route. So swiping from 1 to 5 to 7 would be a route.

I started to look at the number of possibilities for a pattern. The way the algorithm works, you can route between 2 and 9 non repeating dots. So I could route from 1 to 2 and that would be valid. I could also route 1 to 5 to 9. For you non-mathheads out there, this is known as a Factorial problem (which is delimited with an !, so 9! means 9 factorial). So for the number of choices with 2 dots, I have 9! but I only get 2 choices. All those possible 2 digit combinations are in the picture below.

All Choices

So for 2 choices we have 9! with 2 choices. 9 * 8 = 72 choices.

However, after thinking about the problem, I realized that I could NOT select 1 to 9. Why? Because 5 is in the way of 1 to 9. So this immediately removes many choices. Not only could I not route 1 to 9, I could also not route these; 9 to 1, 1 to 3, 6 to 4, etc. It has been a LONG time since my discrete match course, but I couldn’t figure out a way to remove these consecutive patterns in a formula, so I graphed it in excel and highlighted in red all of the ones that would not be possible. The impossible choices are highlighted in red.

CroppedNoChoice

As you can see, by removing the impossible choices, we have reduced our possible combinations by 16 total (which is also 22%) from 72 to 56.

I would also add that their is another subset of choices, those are the “inconvenient” choices. These include all routes that involve moving from a corner to an opposite side middle dot or vice versa, for instance 1 to 6. To successfully do this you have to weave your finger between the 2 and 5 dot while routing from 1 to 6. In a world where convenience is king, I thought it important to consider these as unlikely and do a different data set on them. I highlighted the inconvenient routes in yellow.

CroppedInconveint

Now with highlighting the 16 inconvenient choices, we have chopped off another 22% off the original set and the number of convenient combinations is 40, which is nearly half of our original 72.

I tallied up the number of possible and inconvenient choices in the table below. To read the table, the top row is your first dot choice, and the rows below represent the total number of Next Choices.  So if you start with 1, you have 8 total choices, but 3 of those are impossible and 2 are inconvenient, leaving you with 5 total choices with impossible removed and 3 total choices with impossible and inconvenient removed.

Table of Possible Routes
X 1 2 3 4 5 6 7 8 9
Total Choices 8 8 8 8 8 8 8 8 8
Impossible Choices 3 1 3 1 0 1 3 1 3
Inconvenient Choices 2 2 2 2 0 2 2 2 2
Total Choices – Impossible Removed 5 7 5 7 8 7 5 7 5
Total Choices – Impossible and Inconvenient Removed 3 5 3 5 8 5 3 5 3

At this point, the math is beyond my ability. However, I do recognize some patterns with the graph and in the palindrome of the numbers above. Either way, I brute forced this by whipping up a couple of quick ruby recursive function that determined the number of good, impossible and inconvenient permutations. I will put the in my github account here. If you need help understanding the code please leave a comment.

What you will notice with the results is that the impossible and inconvenient routes compound exponentially, meaning that the more dots we have, the quicker we lose possible good routes. The results are below.

Table of Possible Routes with Impossible Exceptions
Total Choices Total Routes Good and Convenient Good but Inconvenient Routes Impossible Routes
2 72 40 (55.56%) 56 (77.78%) 16 (22.22%)
3 504 160 (31.75%) 304 (60.32%) 200 (39.68%)
4 3024 496 (16.40%) 1400 (46.30%) 1624 (53.70%)
5 15120 1208 (7.99%) 5328 (35.24%) 9792 (64.76%)
6 60480 2240 (3.70%) 16032 (26.51%) 44348 (73.33%)
7 181440 2984 (1.64%) 35328 (19.5%) 146112 (80.53%)
8 362880 2384 (.66%) 49536 (13.65%) 313344 (86.35%)
9 362880 784 (.22%) 32256 (8.89%) 330624 (90.11%)

http://www.stef.be/dev/javascript/patternlock/

But it gets more complicated than that. You CAN go from 1 to 9 if 5 is selected.  So I added exceptions to our algorithm that tests for our impossible routes and then checks the preceding string to see if the dot that would allow the formally impossible string is present.  So if the route were “52197″.  We would see the “19″ and then check the digits before that, since their is a “5″ in “52″ this route is actually possible.  Here are the numbers after adding the new logic.

Table of Possible Next Choices
Total Choices Total Routes Good and Convenient Good but Inconvenient Routes Impossible Routes
ALL 986400 46128 (4.68%) 389203 (39.46%) 596912 (60.51%)
2 72 40 (55.56%) 56 (77.78%) 16 (22.22%)
3 504 176 (34.92%) 320 (63.49%) 184 (36.51%)
4 3024 648 (21.43%) 1624 (53.7%) 1400 (46.30%)
5 15120 2040 (13.49%) 7152 (47.3%) 7968 (52.7%)
6 60480 5248 (8.68%) 26016 (43.02%) 34464 (56.98%)
7 181440 10448 (5.76%) 72912 (40.19%) 108528 (59.81%)
8 362880 15168 (4.18%) 140704 (38.77%) 222176 (61.23%)
9 362880 12360 (3.41%) 140704 (38.77%) 222176 (61.23%)

Conclusion

We find that we have less than 5% of the 986,400 dot patterns are in the subset of Good and Convenient, which is over 46,128 routes. Most phones will only give you 5 tries per 30 seconds so it would take over 64 hours to get through every permutation. To get through all Good but Inconvenient Routes it would take over 540 hours, which is a little less than 22 days.

So, although this is not anywhere close to a game changer, I suspect that this problem will end up being similar to passwords. For instance, although their are 8,153,726,976 possible 5 character passwords using 96 characters, the vast majority of them will be common ones like “12345″, “god” and “monkey”.

I suspect that we will find something similar with pattern swiping. For instance, most right handed people probably start with dot 1 because that is where the thumb is when you pick it up with your right hand. Left handed people will probably have the same tendancy with dot 3. Also, their will also probably be some motions that I did not capture in the inconvenient subsets I defined above.

I also suspect that most people will have a tendency to use a set number of dots, I don’t know what that it, but 4 or 5 seems like something that is convenient and long enough to make someone feel good about setting the swipe password.

Here soon, I will put up a swipe checker and collect data from people for a few months to see what we find.

The code can be accessed here https://github.com/Austio/DotPattern

Changing Show and Edit Actions in Mercury.js for PDF preview

In the mercury-rails/wickedpdf application, I wanted to make a simple additions to limit the pages that people can go to.

I wanted the index page to display a tiny image representation of the pdf they have and then have an edit, pdf preview and destroy option.

I could have done this through a route, but i figured that it is best to just keep it in my index. My table loop is below. The preview just takes the template and appends .pdf at the end. The edit link apends /editor before the template/id so that it opens up the mercury rails.

% @templates.each do |template| %>
  <tr>
    <td><%= template.name %></td>
    <td><%= link_to 'Preview PDF', "/templates/#{template.id}.pdf", :target => "_blank" %></td>-->
    <td><%= link_to 'Edit', "/editor/templates/" + template.id.to_s %></td>
    <!--<td><= link_to 'Edit', edit_template_path(template) %></td>-->
    <td><%= link_to 'Destroy', template, method: :delete, data: { confirm: 'Are you sure, this is not reversible?' } %></td>
  </tr>
<% end %>

Now that part was pretty easy. The next step is fairly straightfoward too. I liked the way the ‘Preview’ button functioned, so I edited that button in mercury.js to look like this.

 preview:                   ['Preview PDF', 'Preview this page'],

Then at the end of our mercury.js file I added an event that binds an action to the preview button

$(window).bind('mercury:ready', function() {
    $('.mercury-preview-button').click(function() {
        alert('foo');
    });
});

Now that this looks like we want and the foo is alerting when we click the Preview PDF button, we just have to change the alert to redirect and
When i click that button after adding the code, foo alerts.

Then change the above code to

$(window).bind('mercury:ready', function() {
    $('.mercury-preview-button').click(function() {
        //grabs the last item in the request URL which is our template id
        var template_id = window.location.pathname.split("/").pop();
        var path = '/templates/' + template_id + '.pdf';
        window.open(path, '_blank')
    });
});

This opens a new window for when the Preview PDF button is clicked that has the PDF representation of our edited document. This feels like a hack, but when i renamed mercury.js to have a .erb and used @template.id I was getting some weird errors that I didn’t want to spend time tracing down. The request ID is coming through in the URL and we already have security in place to prevent URL hacking, so this gives us the functionality we want.

Our final bind event for mercury.js that allows for custom saving and pdf redirection looks like this.

$(window).bind('mercury:ready', function() {
    var link = $('#mercury_iframe').contents().find('#edit_link');
    Mercury.saveURL = link.data('save-url');
    link.hide();

    //opens new page when Preview PDF is clicked in mercury.js
    $('.mercury-preview-button').click(function() {
        //grabs the last item in the request URL which is our template id
        var template_id = window.location.pathname.split("/").pop();
        var path = '/templates/' + template_id + '.pdf';
        window.open(path, '_blank')
    });
});

//redirects after a save
$(window).bind('mercury:saved', function() {
  alert('Template Saved!');
});