June
03
Filed Under (Geekspeak, Work) by Justin on 2008-06-03

If you’ve been following all my posts about SonicWALL, you know that I’ve purchased a ton of gear to construct a nice, widespread VPN. This network consists of the SonicWALL NSA 3500 at my corporate office and a TZ150 at 20 of our remote offices.

Today, I began production deployment to remotes and I have to say, am absolutely, 100% satisfied with the SonicWALL VPN solution. It really is a thing of beauty and it “just works” like you would hope and expect. I’ve had a TZ150 at my house for about a week and I can move my laptop from the office to my coffee table at home, and aside from the latency browsing network shares and such, I still feel like I’m physically connected to the Corporate LAN.

I’ve struggled for the past 12-18 months with deploying our Aastra 9112i VoIP telephones to branch offices for a number of reasons, but primarily because of all the NAT problems associated with SIP packets. The other nagging issue was provisioning and maintaining updates to all these phones once in the field. All the config files and firmware for the Aastras reside on the Asterisk server at the Corporate HQ and are accessed via TFTP and I wasn’t really keen on opening up that port to the entire world. The VPN solves all of these problems! I did the initial provisioning by plugging the phone in to my VOIP LAN at Corporate. The phone pulled down it’s config (which now contains ONLY the internal addresses in the config file) and the latest firmware update as well. Once that was complete, the phone rebooted and I made a successful test call. I took the phone home with me last night, plugged it in to my home network which has that VPN tunnel to Corporate already up, and the phone linked up to Asterisk right away with no additional finagling. Color me impressed!

So now I only have 19 more devices to deploy over the next several weeks and our IT infrastructure will certainly be exponentially more secure than it was a week ago. The VPN is something that has been needed for a while, but funding it was always an issue. Considering the nature of our business, all the personal information we deal with associated with that, and the rising rate of identity theft, we finally realized the time was right and the risk too great to continue operating the way we were any longer. I’ll continue to post updates as the deployment progresses.

(8) Comments    Read More   
June
03
Filed Under (Geekspeak, Work) by Justin on 2008-06-03

Since posting last week about my troubles treading in to SonicWALL water, I think my issues have all been resolved and things are really humming right along. Truth be told, the problem was really a combination of user-error and user-ignorance. If you follow me at all on Twitter, you already know that I had major, show-stopping issues over a couple of days last week. WAN->LAN traffic would flow perfectly fine for a while, and then at some completely arbitrary point, it would stop passing traffic to certain internal hosts. Like I said, this was show stopping.

You might like some background on our implementation. When ESI received our first IP assignment from BellSouth many years ago on our fractional T-1, it was a short and simple /29 network which yielded six usable addresses, minus one that had to be assigned to the ISP provided Cisco router, leaving us with five addresses we could use. A couple years ago (three to be exact), we outgrew that, and in an attempt to keep a 1:1 ratio on NAT rules, we acquired an additional subnet – but this time a significantly larger /28 was assigned. For whatever reason, we never bothered moving all the services to the new subnet. With Watchguard, it was never an issue and was easy enough to run them all simultaneously. The first subnet was configured on the WAN interface and the additional addresses were added as “virtual addresses” on that interface and they were then available to create rules/policies with.

Enough with the history lesson – let’s move on to the present. Personally, I’ll take 99% of the blame for the user-error part. Upon receiving the NSA 3500, I was anxious to get started, so I unboxed it and started exploring the web admin interface. It was confusing and quite different for someone like me, who prior to this, had only had experience with Watchguard gear. After questioning many SonicWALL users whom I respect, I started firing off the Public Server Wizard and creating all my NAT Policies and Firewall Rules. This was apparently mistake number one. While it does indeed work, I later found out that Mark doesn’t suggest that method. Creating them manually gives you a bit more granualar control, leaves a little less cruft in the auto-generated NAT Policies, and the real kicker – you end up gaining a much better understanding of how the Firewall Rules and NAT Policies work together.

Moreno and I made plans to work on this last Thursday morning. I went in to the office early and decided I’d put the SonicWALL NSA back in production while we finished up. Around 7:30 AM, I made the swap and rebooted the Cisco from AT&T to flush any routing related issues out. A few services came online with the NSA and Mark continued cleaning up the rest of the NAT rules and chasing rabbits. By 8:15 or so, we had everything working except one Linux server which was non-mission critical, so we decided to hold-off on that one for a while and see how things went. Things went extremely well throughout the day, but as soon as we finished watching the LOST season finale on Thursday night around 11:15 PM, my Treo started buzzing with down alerts. I started checking things, and finally at this point, it dawned on me that all the services that were failing were on the newer /28 subnet. Surely that must bare some significance. I rebooted the NSA to no avail. After about an hour, traffic magically started flowing to all hosts again so I called it a night, texted Mark to let him know about the issue, and set the alarm for 6:00 AM.

I crawled out of bed around 6:30 Friday morning and got to the office around 7:15. At this point, I had resolved to myself that it was either do or die. I was not going to revert back to Watchguard a third time. If we couldn’t fix the SonicWALL by noon, I was going to wash my hands of it and let Moreno have his gear back. I chatted with Mark around 8:00 and he was bumfuzzled and en route to meet with another client but promised to look at my logs if I’d send them over and also open a ticket with SonicWALL to get the issue resolved ASAP.

I kept hammering away. Sitting idly and waiting on Mark and SonicWALL was not part of my playbook and for some reason, neither was calling support directly. I’m just not the kind of guy most of the time. For some reason, I’d rather spend several  hours resolving an issue on my own than just calling and asking someone. Maybe it’s pride? Regardless, around 9:00 I turned to Google and the SonicWALL website and within 15 minutes, I discovered SonicWALL KBID 3726 titled “SonicOS: Configuring Multiple Subnets Using Static ARP with SonicOS Enhanced” which outlines these simple instructions:

Follow these instructions to create a second subnet on an interface:

  • Create a static ARP assignment. Enable the “publish entry” check box.
    1. Login to the SonicWALL’s Management page.
    2. Select Network > ARP.
    3. Click the ADD button under Static ARP Entries.
    4. IP Address – Specify the IP address to which the SonicWALL should be assigned on the additional subnet.
    5. Interface – Specify the interface (LAN / WAN / OPT / WLAN) where the additional subnet resides.
    6. Publish Entry – Enabling this option causes the SonicWALL to respond to ARP queries for the specified IP address with the SonicWALL’s MAC address. This box must be checked when creating additional subnets.
    7. Click OK.
  • Select Network > Routing.
  • Select Add. Create the following new route policy:
    • Source: ANY
    • Destination: Create new address object
      • Name the object for your secondary subnet
      • Zone Assignment of your secondary subnet
      • Type: Network
      • Network: Enter the Network address of the secondary subnet
      • Netmask: Enter the Subnet mask of the secondary subnet
      • Click OK
    • Service: ANY
    • Gateway: 0.0.0.0
    • Interface: Select the interface the secondary subnet resides on
    • Metric: 20
    • Comment: Label policy so it can be identified at a later date
    • Click OK
  • A ssecondary subnet on the LAN interface will use the default NAT Policy & Access Rules. Access rules & NAT policies may be added.

As soon as I created the ARP assignment, traffic started flowing but I went ahead and created the static route as well for good measure.

I’ve got a couple more SonicWALL posts to come. One about how AWESOME the hardware-based VPN is and another about exactly how to configure the Firewall Rules and NAT Policies without using the Wizard. Tomorrow (Tuesday) morning, I’m heading to Charlotte for the SonicWALL Roadshow and then in the afternoon, I’m deploying my first TZ 150 endpoint to a remote office, along with another special piece of hardware. I’ll try to get some pictures of all that and the VPN post up up by the weekend.

Overall, I’m quite happy with the NSA 3500 and my SonicWALL VPN solution. For now, I’ll withhold my formal review and recommendation on Moreno until I see his final invoice – if he cuts me some slack, I’ll cut him some on the blog.

(6) Comments    Read More   
May
28
Filed Under (Geekspeak, Work) by Justin on 2008-05-28

As previously mentioned, we’re switching away from a one-year old Watchguard Core x750e to SonicWALL NSA 3500 at my place of employment in order to deploy a nice, widespread (geographically speaking), and expensive VPN.

I received the first half of my gear from Mark Moreno two weeks ago and immediately unboxed the NSA. It’s quite a purdy device! It’s sleek, silver, and has a very bright blue LED on the front. I powered it up and upon logging in to the web management interface, I was equally impressed by how shiny and web 2.0 the web UI was. Sadly, that’s where my enthusiasm ends for SonicWALL right now. I started digging around and was just overwhelmed at the options and difference in terminology between the NSA and the Watchguard. After talking it up in the CITRT IRC channel, I was informed that the “public server wizard” was the way to go with configuring NAT policies since SonicWALL  actually needs THREE rules to create one NAT rule. Not only the the NAT policies have to be defined, but then there is the firewall policy. Best I can tell, to NAT one port to one service would require the following steps without the wizard:

  1. Create “Address Objects”
  2. Create “Service” or “Service Group” if not predefined
  3. Create Firewall rule
  4. Create the three NAT policies

While four steps seems simple, it’s a lot of clicking and a lot of digging around, and so far, I’m not a fan. The wizard did a good enough job for some of my rules, but others don’t work right (will work for a few hours and then stop) and others don’t even work at all. At this point, the firewall is doing WAY too good of a job at blocking services from the outside world!

I’m sure it’s a PEBKAC or maybe even an ID ten T error, because so many people just love their SonicWALL stuff. A few minutes ago, I said this in the IRC channel, and I think it’s fairly accurate at a certain level:

<wantmoore> i’d almost go out on a limb and say “windows is to linux as watchguard is to sonicwall”
<DavidSzp>    wantmoore: That’s an interesting analogy
<wantmoore>    watchguard: much easier to do stuff and make it work. sonicwall: a lot more flexibility, but not nearly as straightforward
<stephensflc>    I would totally agree with that statement at this point
<wantmoore>    the analogy doesnt stick where cost is concerned though 😉
<wantmoore>    in that regard, watchguard is a WHOLE lot cheaper. sonicwall will nickel and dime you to death

And I’ll stand by those statements for now. I’m sure that Moreno will help me get my issues resolved and I’ll join the Happy SonicWALL Club soon enough. Until then, I really miss my Watchguard and I’ll be hanging out in the corner with my friend Ed talking about our plans to startup and anti-SonicWALL user group.

(6) Comments    Read More   
July
09
Filed Under (Geekspeak) by Justin on 2008-07-09

I’ve officially crossed the halfway mark in the big SonicWALL deployment of 2008. Ten of the 19 endpoints are in place and working absolutely beautifully. We are deploying two more tomorrow and the final seven on Thursday if all goes as planned. I had a few challenges pop-up, not surprisingly, related to DNS. The solution – a new Ubuntu 8.04 virtual server in VMWare to run bind and do some caching to lighten the load on my Active Directory DNS service.

Full report to come upon completion of project.

(1) Comment    Read More   
June
01
Filed Under (Geekspeak) by Justin on 2008-06-01

I realized shortly after posting about the SonicWALL trouble I’ve been having that three weeks had elapsed since my previous blog post. There’s just been a zillion and three things going on around home and work. Bonnie and I actually have some major, life-changing news to share soon. Many of you probably already know about it, but it’s not totally 100% officialized yet, so I’m withholding talking about it for now in such a public forum. But stay tuned – and please be praying for us in the meantime!

(0) Comments    Read More   
April
19
Filed Under (Geekspeak, Work) by Justin on 2008-04-19

The blogging has been pretty sparse here lately, and I’m quite aware of that fact. Most of the action for me has started happening over on Twitter because its so quick and easy.

I kicked off a major project or three at work this week and they’re all dependant on one central project: a massive VPN rollout. For those who don’t know, I work for smallish company with a rather large footprint. Counting our corporate office where I work, we have 25 locations. Each branch office relies on several services housed at corporate and connect to these sources over the wide-open internet. A VPN has always been on my radar, but never really been considered fiscally until about two week ago when we finally decided to bite the bullet and do it.

My initial plan was to use OpenVPN running on top of Linksys WRT54GL’s at the branches and grab a new Dell R200 to be the hub of the VPN. After about three days of flashing different firmwares on my Linksys at home (OpenWRT, dd-wrt, Sveasoft) and trying to make it work, I threw in the towel and went back to the drawing board. I’ve mentioned before that we have a Watchguard Firebox Core X750e at corporate and I’ve been mostly happy with it. A quick look showed the Firebox Edge X10e to be the cheapest endpoint available to this with Watchguard hardware. However, at around $300 each, this would really quickly get very expensive. Not only that, but I couldn’t come up with anyone I knew who had a Watchguard deployment of this size to ask their advice and opinions.

As I was seeking advice from my pals in the Church IT RoundTable IRC channel, by some act of providence, Mark Moreno decided to grace us with his presence in the channel. I definitely need to insert a disclaimer and an apology at this point. Mark is a guy who is really knowledgeable and passionate about the product he sells and he makes no excuses for being a salesmen either. As a result, I’ve always given Moreno a really hard time about SonicWALL gear, mostly just for kicks. He knows to expect this kind of trash-talk whenever I’m around and always takes the ribbing in good fun. As Mark came in, I made the joke to everyone that I could probably make him drool with the details of the VPN project I was working on. Sure enough, he took the bait and started putting together some quotes. His initial number was somewhere in the range of $15,000 and I just laughed. We talked back and forth over the course of a few days and settled on a new  SonicWALL NSA 3500 to replace the Firebox at corporate and 22 SonicWALL TZ 150 endpoints to go to the branch offices and stripped them down to firmware only – no UTM or support options. The best part of all this is that Mark is preconfiguring all the VPN tunnels before shipping the hardware to me. I really give major props to Mark and SonicWALL for working hard to match an absolutely INSANE price that I found on NewEgg for the TZ 150. The final pricetag with hardware and his consulting time was just a little more than half of his original quote – quite a substantial savings!

Once the VPN is in place, I finally be able to rollout the IP phones I’ve been sitting on for a year to the branch offices, implement a web-based time clock for our staff employees, and join the remote PC’s to our Active Directory domain, which opens a TON of doors for management of these remote computers (software deployment and security patches via WSUS to name a couple).

So, if you don’t hear from me in the next few weeks, just check my Twitter feed (also conveniently located in the sidebar of this blog) or drop me an email or leave a comment. I’ve got a lot of work to do! Thanks again to Mark Moreno for making this a reality from a budget standpoint. I’ll update here as we progress with the implentation and you can bet I’ll let you know if I hit any snafus specifically related to SonicWALL.

(6) Comments    Read More