Hardware ordered August 2004

See also: Wikimedia servers.

The hardware order in August 2004 was 1 NFS fileserver, 1 Search database Server, 8 Apaches, one gigabit ethernet switch, an additional 146GB SCSI drive for Suda (so it could be switched from RAID 5 to RAID 10) and 1 terminal server. The order was modified later that day to change the web server disks to 200GB to allow for more backup, log and temporary space without using the more costly RAID setups for the tasks. It also potentially allows creation of a large virtual drive array, if tests find a network file system which meets the speed and reliability needs of the project. The switch and hard drive were ordered separately from the complete computers. US$17843 was spent.

NFS Fileserver

Transitioning to be the primary photograph server. It's also expected to directly serve database downloads for those mirroring the site and may directly serve images in the future - at present they are sent to and served from the Apache web servers.

Dual Opteron 242 1.6GHz
1 Gb ram (2x512), leaving 6 slots available
3Ware 9500S-8 8 port RAID controller (how much RAM, does it have battery? standard is 128MB, no), expandable to 1GB of cache and battery backup
6x250GB drives (750GB HW RAID 10) actually 5 disk RAID 5 with 982GB available and 1 hot spare
floppy
Sony 52x CDROM IDE
460W hotswap redundant power supplies
Fedora Core 2 for AMD64

Search DB server

To offload search from the primary database server and allow more efficient memory use, because search uses MyISAM tables while the rest of the database uses InnoDB and the two can't share the same cache RAM. Splitting it allows each to use all of the RAM most efficiently.

same machine as above, except...
6x200GB drives (600GB HW RAID 10)
4 GB ram (4x1gb), leaving 4 slots available

Apaches

These temporarily removed the page building/web serving part of the system as the primary bottleneck. They should provide sufficient page building capability until November or December 2004.

8 1U Pentium 4 machines.

P4 3.0GHZ - 1 MB Cache - HT - 800FSB
512MB (2x256) - leaving 2 slots available
200GB ATA100 hard drive

Gigabit switch

A 24 port Netgear GS724T gigabit switch will help to improve load balancing, reduce latency for queries being passed between the computers in the cluster and remove the 100 megabit per second limit for the busy outgoing connection. It has two generic SFP GIBC fiber ports for uplinks, which each remove one non-fiber port if used.

Default configuration is to negotiate only 10/100 speed. Default IP (without a DHCP server on the network) is 192.168.0.238. As a smart, but not managed, switch, it can set priorities, create VLANS and be managed remotely via a web interface but does not have SNMP support.

Email announcing the order

[Wikitech-l] Big new server order under way
Jimmy (Jimbo) Wales jwales at wikia.com
Wed Aug 4 19:52:32 UTC 2004

    * Previous message: [Wikitech-l] de.wikipedia in utf-8
    * Next message: [Wikitech-l] Big new server order under way
    * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

I have ordered some new servers.  I believe that they will resolve
several significant issues.

1. NFS Fileserver -- for now, only 1 of these, as I was told that
gwicke wants to do some tests before we buy 2 of them.

Dual Opteron 242
1 Gb ram (2x512), leaving 6 slots available
3Ware 9500S-8 8 port RAID controller
6x250GB drives (750GB HW RAID 10)
floppy
Sony 52x CDROM IDE
460W hotswap redundant power supplies
Fedora Core 2 for AMD64

2.  Search DB server -- 
same machine as above, except...

6x200GB drives (600GB HW RAID 10)
4 GB ram (4x1gb), leaving 4 slots available

3.  Apaches --

In what may be a slightly controversial decision, but supported by
Shaihulud and Jeronim in the irc room, and probably a tossup to most
people, I went with a "known quantity" and ordered 8 1U Pentium 4
machines.

P4 3.0GHZ - 1 MB Cache - HT - 800FSB
512MB (2x256) - leaving 2 slots available
80GB ATA100 hard drive

My thinking here is that this will roughly *double* our apache
capacity, thus definitively solving (for now) what has been a
longstanding issue.  This may expose something else as a bottleneck,
we don't know yet.

There was a lot of talk about going with dual opterons in this
capacity, and it was once decided.  But lately there has been some
concern about the performance of suda, and some difficulties
(possibly) with having a non-homegenous group of apaches.

I have seen arguments back and forth, and did not find any of them
definitive.  So, I made a decision.  That's what I have to do, and I
hope no one is mad about it.  :-)

--Jimbo

Vendor's description of the switch

To preserve the history of the project beyond the time the product is being sold, here is a copy of the vendor's description of the gigabit switch at the time of the order:

"Key Benefits
24 10/100/1000 Mbps gigabit ports
Non-blocking bandwidth
Easy, browser-based switch management
Rack mountable
2 Small Form-factor Pluggable (SFP)(1) GBIC slots
(1)Use with NETGEAR AGM731F 1000BASE-SX SFP GBIC module or NETGEAR AGM732F 1000BASE-LX SFP GBIC module.

Management at an Unmanaged Price
Joining NETGEAR's family of Smart Switches with simple switch management at no extra cost, this 24-port Gigabit switch is high on value and capability. With 24 10/100/1000 Mbps ports, each capable of powering 2000 Mbps of data throughput in full-duplex mode, plus two optional fiber ports, it provides fast backbone and connects servers or power users, delivering large amounts of multimedia, image, and video information in no time at all. Ideal for combining 10, 100, and 1000 Mbps devices it delivers 10/100/1000 automatic speed and full/half-duplex sensing plus Auto Uplink» on every port. Two hot-swappable Small Formfactor Pluggable (SFP)** GBIC slots provide optional fiber connectivity for greater distance. An intuitive, web-browser interface offers simple switch management, making it a snap to monitor switch performance, configure ports, even set up trunks, VLANs, and prioritization.

High Powered
Twenty-four 10/100/1000 ports deliver non-blocking, full-speed packet forwarding with 48 Gbps bandwidth migrates your network to future-proof functionality. Two SFP GBIC fiber slots provide optional fiber connectivity for greater distance.

Intelligent
Smart Switch capability enables key management features for the value price of an unmanaged switch. Included are performance monitoring, port configuration, VLAN for traffic control, port trunking for increased bandwidth, and Class of Service (CoS) for traffic prioritization.

Right Sized
Ideal for growing businesses who want the benefits of management without the cost and complexity of a fully SNMP-capable network.

Easygoing and Durable
Simple to set up and simple to use for effortless network monitoring and troubleshooting. Made to last, this sturdy switch is housed in a 17-inch metal box and comes with a rack mounting kit."