One of the things that came out of our rails bugmash team and time at Blue Box Group was that the real time killer was a lot of the prep work. Installing 3 databases, and all of the other tools and goodies can take hours which could be used mashing bugs and racking up points.More …
I have run into this three or four times now. Each time takes some googling, and that is counterproductive. So here you go. For all of the times you get a new mac, or install a new version of OSX, and you start seeing weirdness in terminal.app while SSH’d into a *nix box..More …
Several things have happened over the last few months that gave me pause. Pause is a great time to practice mindfulness.
A few days ago, Les Paul left us. Walter Cronkite too. I was laid off from my job, and I got an exciting opportunity presented to me. Change has been pretty rampant. However the curious case of a hacker named _why has hit me closest to home.
Just yesterday _why left us. All of his digital presence is gone. Code, Writing, Songs, Chunky Bacon. “But, wait, who is _why, and why do we care about _why?” you ask.
Well, I guess you are not a rubyist. He was one of three luminaries in the community, and one of the reasons that many of us got involved. He wrote the greatest programming book ever (with a sound track!), and coined “Chunky Bacon”. DHH gave us “and, Woops” and matz the language itself. But _why gave us the quirky character. The oddness. The wonder and joy. Being ok to look at some code and go “wow, that’s cool” with a childlike glint in our eyes.
One day the Buddha was sitting in the wood with thirty or forty monks. They had an excellent lunch and they were enjoying the company of each other. There was a farmer passing by and the farmer was very unhappy. He asked the Buddha and the monks whether they had seen his cows passing by. The Buddha said they had not seen any cows passing by. The farmer said, “Monks, I’m so unhappy. I have twelve cows and I don’t know why they all ran away. I have also a few acres of a sesame seed plantation and the insects have eaten up everything. I suffer so much I think I am going to kill myself.
Will he come back? Who knows.
His exit has caused quite a stir in the Ruby community. But can the Ruby community really be defined by one, two or three people?
I’d say no. _why gave us a wonderful sense of joy and childlike zeal to the community. Just because _why left, why do we need to loose that? After all, _why was a character in all of the definitions of the word.
I’d also like to take a second to lay to rest a misconception. _why does not owe anyone anything. Did any of us pay him for ongoing support? We befriended a character, and that character’s story is over. Do Sam Spade and Luke Skywalker owe us anything? No. It’s silly to even think like that.
The Buddha said, “My friend, we have not seen any cows passing by here. You might like to look for them in the other direction.” So the farmer thanked him and ran away, and the Buddha turned to his monks and said, “My dear friends, you are the happiest people in the world. You don’t have any cows to lose. If you have too many cows to take care of, you will be very busy. “That is why, in order to be happy, you have to learn the art of cow releasing (laughter). You release the cows one by one. In the beginning you thought that those cows were essential to your happiness, and you tried to get more and more cows. But now you realize that cows are not really conditions for your happiness; they constitute an obstacle for your happiness. That is why you are determined to release your cows.”
So, _why, thank you. If you ever need anything, you know where we are. We love you. Have a good journey.
As a test I was asked the following question
“describe how the internet works. Be detailed. Ignore the physical layer. You have 1 hour and no reference materials.”
Wow. It was kind of fun though. Below the jump is How The Internet Works by Me. Typos and all. Total stream of consciousness writing. Honestly, reference materials would have just slowed me down.
If you think it’s easy, get out a timer, and go - just don’t read mine first. ;-)
How the internet works:
At a very high level the internet is made up of lots of little networks which have agreed to use a common protocol and addressing scheme. This allows messages to travel from network to network - and just about any host (a host is a computer, server, router, printer - anything with an address) on the internet to talk to just about any other host.
Now, that’s all well and good, but there are several major flaws with the system described above. First off, if anyone can talk to anyone, then bad guys are able to do bad things to hosts they should not have access to (like ATM’s). Secondly, there are a finite number of addresses in the addressing scheme that was developed way back in the stone ages.
We’ll talk about the flaws in a bit. First we need a basic understanding of the lingo.
Host (as described above) is a computer, printer, router, or anything else that talks on the network (iPhones…)
An IP address is just like a postal address - and you read it very similarly. It looks like XXX.XXX.XXX.XXX with the last number being the most specific and the first being the most general. Like 1234 anystreet, anytown in reverse.
A MAC address is the physical address of a network card/device. IP addresses are translated to MAC addresses late in the game.
A router reads the IP address and determines if it needs to send the message along to another network. For example, if you are on the 10.10.0.0 network and you are sending a message to 192.168.0.45 - you will need to use a router to pass the packets between the subnets. Routers use Routing tables to find other networks. Since having a table of every network on the internet would be to big for a router to parse rapidly, we use several protocols for routers to determine who is next in line on the trail to the destination. Some of those protocols have odd names like OSPF, BGP or RIP. Misconfigured routers can cause all kinds of headaches including the dreaded “routing black hole” (insert scary music).
A subnet is a logical IP address grouping that might map to a physical network. It’s defined by the Subnet Mask. An IP address might be 10.10.0.2/255.255.255.0. that says that the host is only defined by the last octet (number) of the IP address (the 2). The rest of the address (the 10.10.0) defines the network. Notice that their is a distinction between 10.10.0.0 and 10.11.0.0 and 10.10.10.0 - all are different networks. Supernetting is used to fuse two or more subnets together into one network (this is used when you have several ip address ranges, and you don’t want the overhead of routing between them)
A gateway is the router that handles traffic in and out of the network that you live on.
A Firewall is a special type of router that examines messages (packets) to insure that they are not from bad guys. Firewalls also will disallow traffic to and from certain hosts, or only allow traffic in response to a request from an internal host. Many firewalls also do NAT.
NAT allows us to map internal IP addresses to external. You can do a one to one mapping (each internal address has a corresponding external address), or many to one mapping (several addresses internally share the same external address). NAT can cause problems with certain internet protocols - and some protocols have been developed which can traverse NAT routers with variable success.
Anyways, now that we have that out of the way, we can press on and talk about the two main issues. We are quickly running out of those addresses - so there is a new version of the addressing scheme that is being touted as the new hotness. Not many people use it yet, however it is slowing growing. One of the hindrances to adoption has been the universal use of NAT. What NAT does is allows IP addresses to be mapped from one to another. Now, there are several ranges of IP addresses which were declared “off limits” by the great grey beards in the sky back in the stone ages, so they cannot cross the internet. Properly configured routers will simply discard (drop) packets that are to or from such an address. NAT allows us to map these “internal” addresses to an external “internet” IP address.
Most SOHO gateways use NAT these days. It allows many internal machines to share 1 or a small number of external addresses. This has relived the burden on the IP Addressing space somewhat. Now not every machine needs an internet IP address - just some of them.
However, we are rapidly approaching “peak IP”, when the last IP address will be handed out. IPv6, the new hotness, will allow for a quantum leap in addressing space, and hopefully we will not need IPv7 for many many years.
One of the other major issues with the internet is (remember that scary music? yeah - play it now) Bad Guys. Bad Guys can do all sorts of nasty things, but the most common today seems to be taking a machine with an OS from the seattle area, inserting evil code, and using it for Distributed Denial of Service attacks, spam generate, phishing attacks and other bad things. They are not called Bad Guys for nothing!
There are several things that can be done to keep the Bad Guys at bay. The first, is to properly secure your system before going out on the information superhighway. Patches, firewalls, Antivirus scanners are all a good idea. A better idea is to run a system that was designed with security in mind (*nix comes to mind) so you have less to worry about.
Internet based applications are another story entirely. Their issues are ones of scale, and traffic. If you get popular, how do you deal with all of the traffic? This is a harder problem then security and IP addressing ranges.
I actually wrote a paper at my last job detailing out a high level overview of scaling web applications. Basically, it comes down to this: Break your application into logical parts and make them do their job with as much speed as possible. Add hardware to the places that need it.
Virtualized hardware is a huge win here, as it can be added very quickly. Being able to spin up “servers” quickly (minutes or hours instead of days) and put them in the systems that need them in response to detected issues is a game changer.
Load Balancing (LB) is also huge. Many LB’s use NAT - so you have 1 IP address outside accepting requests, and they use some logic to split the traffic between several machines behind them. You can layer your LB’s even
In an extreme case it might look something like this:
Internet –> LB –> Content Servers –> LB –> Application Servers –> LB –> Database servers
The internet traffic hits the external LB, and is split into several content servers. If the content is a hard asset (a picture, or a cache item) it’s simply delivered back to the LB and sent on it’s merry way. If not, then the request hits another LB (which might be built into the content server) and the traffic is routed to a not so busy application server. If the app server needs to make a DB call, then the request hits another LB, and is routed to a DB that can handle the request and that is not busy.
In the past I have found that Nginx is a great content server. The application server depends on the back end application (mongrels/thin/passenger for rails, tomcat etc for java…), and not all DB’s support load balancing (at least for writes).
You need to have DB, App and Asset caching in there somewhere. The basic rule of thumb is that if you have to hit disk you are dead in the water. Disk access can add hundreds of milliseconds to a request. It a border line system or during a traffic spike requests can start to queue up behind the slow requests, and you have unhappy customers.
On of the tasks that I have undertaken is helping out a non-profit called Railsbridge by increasing community awareness. They do a ton of really cool things - like Rails Workshops for Women (one upcoming in San Francisco!), Rails Mentoring, Student Code Reviews, and undertaking development efforts for worthy Non Profit organizations.
So, stay tuned - good stuff is going to come of this little project. We tend to hang out in #railsbridge on freenode, so if you want to learn more, or you want to see more, head on over, and I’ll see you there!