Rolling Your Own NAS with FreeNAS: Part Two

In my last post, I described my motivation for building a home brewed NAS. I talked about why I opted against some of the more commercial alternatives. And I briefly mentioned a few of the perks of having a FreeNAS solution for your home or office.

In part two, I’d like to discuss my hardware choices and touch on some of the things that might influence your own FreeNAS project.

The Case

The case was the very first place I started on this project. It might seem like an unexciting beginning. But the choice of case has the potential to widen or limit the selection of hardware available to you when building your machine.

For my server, I wanted something small, something almost toaster-sized. The server would be living in my office, a room severely constrained in terms of space. So I needed something that I’d be able to tuck out of the way. And I also wanted something with room for at least two hard drives that I’d use in a RAID configuration.

Below is a picture of the case I settled on. It’s the PC-Q11B from Lian Li. It’s a Mini-ITX tower with room for a motherboard, power supply, two hard drives, and little else. It’s bigger than my toaster, but not by much. It’s sleek and everything about it is cleverly designed.

It’s worth noting that if you decide to purchase this case, it accepts a full ATX style power supply and not a Mini-ITX style.

The Motherboard and CPU

Next on my shopping list was the motherboard and CPU. This was a little tricky. Picking components for a Windows PC is easy since every PC hardware manufacturer has Windows drivers. But FreeNAS is a FreeBSD based OS. So it requires a little bit of homework to find a motherboard/CPU combo that’s fully supported by FreeBSD.

The two things I was most nervous about was the USB and network interface support. I had planned on running FreeBSD from USB stick (more on this in part 3). And, of course, if the network interface built into the motherboard wasn’t supported, the server would be of no use. So I spent a couple of days scouring the various forums, mailing list archives, and hardware compatibility lists.

In the end, I purchased a BOXDQ45EK LGA 775 motherboard from Intel along with a Pentium E2160 processor. And to be perfectly honest, this purchase was a shot in the dark. I didn’t really know if what I was getting would work well at all. But with Intel components, I figured the odds were in my favor. And as it turns out, they work pretty well.

  

In addition to the processer and motherboard, I picked up a Fanner-420 heatsink/fan made by MassCool.

Memory

FreeNAS doesn’t need a lot of memory. You could get by easily with 512MB of RAM. But RAM is dirt cheap. So I purchased 1GB of Crucial sticks.

Power Supply

I goofed on the power supply. Both the case and the motherboard I wanted to use were Mini-ITX. So I naturally reasoned that I needed a Mini-ITX power supply. I initially bought a 220W Mini-ITX power supply from FSP Group. It was only after I began assembling everything that I realized I had made a mistake. The Lian Li case doesn’t support Mini-ITX form factor power supplies, only ATX.

The FSP got shipped back to NewEgg and I picked up a 400W Rosewill RD400-2-DB instead.

Hard Drives

The primary goal of this project was to provide a centralized location for shared media like photos, home video, etc., as well as to provide a location where the other machines could stash backups. Based on our history, I estimated that we’d probably use less than 1 TB of storage over the next few years. So I opted for Western Digital’s Caviar Green WD10EZRX 1TB drives. I purchased two of them that I would end up using in a simple RAID configuration. WD’s Green drives aren’t going to win any races. But they’re low power. And for a device that’s always on but barely used, powers savings is important.

Extras

While it’s not directly part of the server, I should mention I also picked up a CyberPower CP1000AVRLCD UPS. We live in an area with frequent power outages and brownouts. It’s important that the server is protected and that it’s able to shut itself down cleanly when a power outage occurs. As it happens, FreeNAS has built in support for UPS devices. It’s a great feature that I’ll have more to say about in part 3.

Total Cost

The total cost for all of the hardware was as follows:

Case $99.99
Motherboard $54.99
CPU $33.99
Heatsink $11.99
Memory $23.99
Hard Drives $191.98
Power Supply $34.99
Thermal Grease $4.49
4 GB USB Stick (for OS) $4.99
UPS $122.99
Grand Total $584.39

All in all, it wasn’t that expensive. A comparable off-the-shelf solution would have cost me a bit more.

Conclusion

We’ve been running this little server for almost 2 months now. And I’ve been very happy with the hardware. I suspect there’s a bit more tweaking I can do to bring down the power consumption. But all in all, this device performs well and does what it was meant to do.

I should mention one thing that I find to be a minor annoyance. Both the server case and the UPS have ridiculously bright blue LED lit power buttons, which you can see below. These devices sit next to each other behind my office door. At night, the whole room casts a blue glow that can be detected in many parts of the house. If you decide to use either of these components, you’ll probably want to keep them out of rooms that need to be dark (e.g., bedrooms, rooms used for watching movies, etc.). Otherwise, your eyes will feel like they’re trying to crawl out of your skull.

In part three…Installing the Software.

Rolling Your Own NAS with FreeNAS: Part One

Once upon a time, managing data in my house was a no-brainer. I had two computers and a couple of external hard drives. I knew where everything was. If I needed to move or copy files from one computer to another, I’d simply create a Windows share. Backups were a manual process, but not so onerous that they didn’t get done. And because I lived alone, I didn’t have to think about things like sharing data with other people.

Then the girlfriend moved in. And with the girlfriend came all of her digital devices. Suddenly photos and videos needed to be shared, documents needed to be accessed from a slew of different devices, and backups became more of a chore. Overnight, my little office, which used to be a digital island of sorts, became assimilated into a larger ecosystem of devices and content. And my tiny, simple network became a lot more complicated.

Something had to be done. And I was fairly certain a significant part of the solution involved dedicated network storage. Thus began my quest for finding the ultimate “one size fits all” data hub for my home network. I examined a number of different solutions including cloud storage, off-the-shelf NAS devices, and network routers with USB ports for external storage. Heck, I even entertained the idea of dual-purposing my own PC for this task (crazy talk, I know).

None of the commercial options really appealed to me. Each had their own drawbacks. And the more I learned about them, the more disappointed I felt.

My must-haves were:

  • The device should be small.
  • Power efficiency. If the device isn’t in use, it must consume as little power as possible.
  • There must be no software to install on client machines.
  • Support for RAID is a must. This also means that internal drives must be easy to access and replace.
  • Client machines should be able to access the device using plain ole vanilla Windows shares.
  • If the power goes out, the device should be able to shutdown cleanly. I live in a somewhat rural area prone to power outages and brownouts.

My like-to-haves were:

  • FTP support.
  • User account support.
  • The ability to remotely administer the device (shell access).

Eventually, I decided that building a custom NAS was the only way I was going to be satisfied. Instinctually, I began thinking about building a simple Linux machine. And that’s the path I actually started down. I wasn’t sure which distro to use. And there would no doubt be a fair amount of configuration, testing, heartburn, and headaches. So I began searching for folks who had rolled their own Linux-based NAS with the hope of finding a few war stories, helpful pointers, and perhaps even hardware suggestions.

It wasn’t long before I came across this wonderful thing called FreeNAS. It was a ready-made platform for folks like me trying to roll their own NAS. It had everything on my wishlist and seemed extremely easy to get up and running.

FreeNAS runs on top of FreeBSD in fact. Not Linux. I have to confess that I’d never used FreeBSD before undertaking this project. And I was a little nervous about the idea. Moving from a Linux distro you’re comfortable with to some other *nix platform can be scary – files are in different places, daemons are managed differently, package management makes no sense, and the arguments to your favorite command line tools behave oddly. My needs going into this project were fairly modest and now I would be tasked with learning an entirely new OS. It seemed a bit much. And I wasn’t entirely sure I was ready to invest a whole lot of time into this project. But as it often happens to geeks and do-it-yourselfers, I was seduced by the sexiness of something new and shiny. And so it began.

Before I go into the details of my FreeNAS project, I would like to briefly discuss some of the options I mentioned previously and the reasons why I chose not to use them.

Cloud Storage

Cloud storage is all the rage at the moment. Services like Carbonite, Google Drive, and Amazon Cloud Drive allow you to store data online and access that data anytime and from anywhere, so long as you have Internet access that is. Some services market themselves as backup solutions, while others sell themselves as more generalized storage solutions. Some services even tout their wares (pun intended) as a means to share files with others. Whatever your online storage needs and budget, there’s undoubtedly a company out there that can help you.

Cloud storage wasn’t a solution for us for two reasons:

Cloud storage simply isn’t practical for big files. Transferring anything over a few hundred megabytes across the interwebs is time-consuming and potentially very expensive. Sharing things like raw DV video or project files from a recording session (did I mention I’m a musician?) isn’t really appropriate for these products. Not to mention, I’m fairly sure if I started transferring terabytes of data back and forth every month, my ISP would start sending me hate mail and would mostly certainly cut me off.

The consumer also needs to be concerned with security when it comes to these services. How much do you trust your storage provider? Are you uploading sensitive data? How silly would you feel if your tax return suddenly became public domain? And what if your storage provider went out of business tomorrow?

Off the Shelf NAS

There are a lot of great consumer NAS products on the market. And they vary widely in terms of features and price.

At the low end (under $250) are products with extremely limited sets of features. They tend to lack RAID support, require client software to be installed on your PC, and they often don’t integrate well with UPS devices. Some even require an account with the manufacturer’s online service to take advantage of features.

As the price point increases, the feature sets are richer and the products begin targeting more professional users. I did manage to find a couple of devices that would have worked well for me. But they were way outside of my budget. I popped over to NewEgg and created a rough estimate of the kind of hardware I thought I’d need. My estimate for a device with similar capabilities and capacity came in at a fraction of the cost as some of these higher end devices.

Routers With USB Ports

Many modern routers come equipped with a nice little USB port on the back. This allows you to plug a mass-storage device (e.g., external hard drive) into the router and make the drive available to the rest of your network. You may or may not have to install software on your PC to use it. It depends on the router and the mass-storage device.

This is simply brilliant. I love the simplicity of the idea. And I run into plenty of situations where something like this would be handy. For me, however, it wasn’t a real solution. Or at least it wasn’t a solution for this particular problem. Apart from the actual storage, these devices lack just about every other feature on my wishlist.

Enter FreeNAS

FreeNAS is a special distribution of FreeBSD that is preconfigured to function solely as network attached storage. All you need to do is provide the hardware, install the software, and configure it appropriately. It supports RAID, UPS integration, all of the popular network file system protocols, FTP, SSH, email notifications, user access control, SMART monitoring, and rsync, and the proverbial kitchen sink. It also provides a really nice web interface for remote configuration. And did I mention it’s free?

In the next two articles, I’ll show you what hardware I chose and the process I used to configure FreeNAS for my home network. You’ll learn how I set everything up and devised an automatic backup solution.

Stay tuned! In the meantime, visit FreeNAS.org!

AsyncTask Missteps

AsyncTask comes up in conversation quite often around the office, usually because of a failed code review or some related bug that’s been discovered. It’s one of the most well understood classes on the Android platform, and yet it’s also one of the hardest classes to use correctly. Even advanced Android developers get it wrong.

In the following discussion, we’ll take a look at a few snippets of code, touch on some of their problems, and arrive (hopefully) at a pattern of AsyncTask usage that’s suitable for most cases.

Let’s start with a fairly typical example.

public class MyActivity extends Activity 
{
    @Override
    public void onCreate(Bundle savedInstanceState) 
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
 
        // Find views and assign them to member variables.
 
        new AsyncTask<void, void,="" string="">()
        {            
            @Override
            protected String doInBackground(Void... params)
            {
                // Do some long running task.
                return result;
            }
 
            @Override
            protected void onPostExecute(String result)
            {
                // Update UI
            }
        }.execute();
    }
    // Various View member variables.
}
</void,>

This seems fairly straightforward. When our Activity is created, we launch an AsyncTask to perform some background work, which is performed in doInBackground. When the background work is finished, doInBackground returns a String which is used by onPostExecute to safely update the UI.

We’re using the AsyncTask here exactly as I’ve seen it used in production code. UI elements are being updated safely. Work is being done in the background. We’re in no danger of an ANR message. So what exactly is the problem here?

The problem isn’t what happens inside the AsyncTask. The problem is what happens outside of it. Specifically, what happens if the Activity is destroyed while the AsyncTask is running?

Imagine this scenario – the Activity is created, the AsyncTask is instantiated and executed, and then the screen orientation changes. When screen orientation changes under normal circumstances, the original Activity is destroyed and a new one is created. This means another AsyncTask will be instantiated and the background work will start all over again.

But that’s not the worst of it. The original AsyncTask will continue to run until it’s finished. And once it’s finished, it’ll update UI elements that aren’t visible anymore. And because the original AsyncTask still has an implicit outerclass pointer to the original Activity, the original Activity can’t be garbage collected until the original AsyncTask is finished. Change the screen orientation a bunch of times on a resource intensive Activity and you risk running out of memory.

There’s also a limit on the number of AsyncTasks you can have running at once. For Activities with smaller footprints, you’ll most likely run out of AsyncTasks before you start to see memory problems appear.

Does that make you uncomfortable? It certainly should.

So how do we work around these issues? Let’s address these one at a time, starting with the simplest – the implicit pointer to the outer class. This can be solved by making the AsyncTask class static, which also means it can no longer be anonymous.

Removing Implicit Outerclass Pointers

public class MyActivity extends Activity 
{
    @Override
    public void onCreate(Bundle savedInstanceState) 
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
 
        // Find views and assign them to member variables.
 
        MyTask task = new MyTask();
        task.execute();
    }
 
    static class MyTask extends AsyncTask<void, void,="" string="">
    {
        @Override
        protected String doInBackground(Void... params)
        {
            // Do some long running task.            
            return result;
        }
 
        @Override
        protected void onPostExecute(String result)
        {
            // Need to update UI. But how?
        }
    }    
    // Various View member variables.
}
</void,>

So far so good. Our AsyncTask will no longer keep a reference to the Activity. But now we can’t update the UI when the task completes, which kind of defeats the purpose. How do we fix this? By giving back an Activity reference to the AsyncTask. “Wait, isn’t that what we tried to eliminate by making the class static?” Well, yes. But we wanted to remove the implicit reference, which is something we couldn’t change at runtime. With something a bit more explicit, we have direct control over when the reference comes and goes.

public class MyActivity extends Activity 
{
    @Override
    public void onCreate(Bundle savedInstanceState) 
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
 
        // Find views and assign them to member variables.
 
        m_task = new MyTask();
        m_task.m_activity = this;
        m_task.execute();
    }
 
    @Override
    public void onDestroy()
    {
        super.onDestroy();
        m_task.m_activity = null;
    }
 
    static class MyTask extends AsyncTask<void, void,="" string="">
    {
        @Override
        protected String doInBackground(Void... params)
        {
            // Do some long running task.            
            return result;
        }
 
        @Override
        protected void onPostExecute(String result)
        {
            if (m_activity != null)
            {
                // Update UI
            }
        }
 
        MyActivity m_activity = null;
    }
 
    private MyTask m_task = null;    
    // Various View member variables.
}
</void,>

That’s better. In the above example we remove the Activity reference in the AsyncTask when the Activity is destroyed. The AsyncTask can continue to run without preventing our Activity from being garbage collected.

Concurrent AsyncTask Limitations

Now let’s briefly discuss the problem of concurrent AsyncTasks. If you weren’t aware that there’s an upper limit to concurrent AsyncTasks, Google “AsyncTask limits” and you’ll find plenty of information about it. We need to control the lifetime of our AsyncTask instance. What that boils down to is canceling the task and letting the garbage collector deal with it. Canceling a task is a two-step process. First, we set the AsyncTask’s cancelled flag in the Activity’s onDestroy method.

m_task.cancel(false);

This alone doesn’t actually cancel the task. It has no effect on the background process if the task chooses to ignore it. So we also need to regularly check the return value of isCancelled() inside of our AsyncTask’s doInBackground method and bail out appropriately. This allows us to terminate that background process cleanly.

if (isCancelled())
{
    return null;
}

Notice that we passed false to the AsyncTask’s cancel method. If you pass true, it interrupts the thread and can leave things in a particular awkward state. See the Thread class’s interrupt method for details.

One AsyncTask To Rule Them All

We still have a problem of a new AsyncTask being created and executed every time our Activity is destroyed and recreated. In most cases, creating a new AsyncTask instance is unnecessary. We often only need the AsyncTask to query a database, make a web service request, or read some file once. So it’d be nice if we could leverage a single AsyncTask no matter how many times our Activity is temporarily destroyed/created.

I’ve seen some folks set the android:configChanges attribute in the Activity’s XML declaration inside the app manifest just to avoid the redundant AsyncTask issue. And while it’s certainly doable, it’s somewhat of an obscure solution. android:configChanges is meant to solve other problems. That approach may confuse other developers who will need to maintain your code. It also makes the UI a bit inflexible because the default behavior may actually be behavior you want.

I’ve also seen some folks address the issue using invisible Fragments. Again, it works. But it’s somewhat obscure if you’re using a Fragment oriented approach just to solve this issue. Make sure your Fragment usage is appropriate for your UI design. Never use a sledgehammer when a rubber mallet will do.

The approach I like best is mentioned in this StackOverflow thread and in Hervé Guihot’s book “Pro Android Apps Performance Optimization”.

public class MyActivity extends Activity 
{
    @Override
    public void onCreate(Bundle savedInstanceState) 
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
 
        // Find views and assign them to member variables.
 
        m_task = (MyTask) getLastNonConfigurationInstance();
        if (m_task != null)
        {
            m_task.m_activity = this;
        }
        else
        {
            m_task = new MyTask();
            m_task.m_activity = this;
            m_task.execute();
        }
    }
 
    @Override
    public void onDestroy()
    {
        super.onDestroy();
 
        m_task.m_activity = null;
 
        if (this.isFinishing())
            m_task.cancel(false);
    }
 
    @Override
    public Object onRetainNonConfigurationInstance() 
    {
        return m_task;
    }
 
 
    static class MyTask extends AsyncTask<void, void,="" string="">
    {
        @Override
        protected String doInBackground(Void... params)
        {
            // Do some long running task. We need to make sure
            // we peridically check the return value of isCancelled().
            return result;
        }
 
        @Override
        protected void onPostExecute(String result)
        {
            if (m_activity != null)
            {
                // Update UI
            }
        }
 
        MyActivity m_activity = null;
    }
 
    private MyTask m_task = null;
 
    // Various View member variables.
}
</void,>

The key piece here is the use of the onRetainNonConfigurationInstance and getLastNonConfigurationInstance methods to reuse an AsyncTask instance between build up and tear down of the Activity. Using this approach will always result in a single AsyncTask being created and used.

My code above differs a bit from that mentioned in the StackOverflow discussion and in Guihot’s book. One notable difference is that I check to see if the Activity is actually finishing in the onDestroy method. If it is, we cancel the task. isFinishing will return true if this Activity is being dismissed as a result of going backwards in the Activity stack.

Our Final Approach

There is still a bug here. Can you spot it? What happens if the Activity has been destroyed and recreated after the AsyncTask has completed? Our UI doesn’t get updated. The solution to this is fairly simple and illustrated in the final version of our Activity below.

public class MyActivity extends Activity 
{
    @Override
    public void onCreate(Bundle savedInstanceState) 
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
 
        // Find views and assign them to member variables.
 
        m_task = (MyTask) getLastNonConfigurationInstance();
        if (m_task != null)
        {
            m_task.m_activity = this;
            if (m_task.m_isFinished)
                m_task.updateUI();
        }
        else
        {
            m_task = new MyTask();
            m_task.m_activity = this;
            m_task.execute();
        }
    }
 
    @Override
    public void onDestroy()
    {
        super.onDestroy();
 
        m_task.m_activity = null;
 
        if (this.isFinishing())
            m_task.cancel(false);
    }
 
    @Override
    public Object onRetainNonConfigurationInstance() 
    {
        return m_task;
    }
 
 
    static class MyTask extends AsyncTask<void, void,="" string="">
    {
        @Override
        protected String doInBackground(Void... params)
        {
            // Do some long running task. We need to make sure
            // we peridically check the return value of isCancelled().
            return result;
        }
 
        @Override
        protected void onPostExecute(String result)
        {
            m_result = result;
            m_isFinished = true;
            updateUI();
        }
 
        public void updateUI()
        {
            if (m_activity != null)
            {
                // Update UI using m_result
            }            
        }
 
        // These should never be accessed from within doInBackground()
        MyActivity m_activity = null;
        boolean m_isFinished  = false;
        String m_result = null;
    }
 
    private MyTask m_task = null;
 
    // Various View member variables.
}
</void,>

I don’t claim that this pattern is a one-sized fits all solution. But I think it covers most cases quite well. There are a few things I avoided discussing here, such as dealing with progress dialogs and how to design tasks that can be used by multiple activities. I didn’t want to add too much complexity to the examples. And I think those sorts of things should be fairly intuitive once you’ve achieved a certain degree of comfort with AsyncTask