Best of the ISP-Lists

The Office Park

This company wants to connect a few customers within the building, making money by reselling a fat pipe. But even in Taiwan, the nation that's the source of so much of our hardware, the telco and other big providers are not friendly to potential competition.

[September 11, 2007]

On the ISP-Tech list in August, K wrote from Taiwan:

Hi! My company is thinking about sharing our internet connection with a few other companies, and maybe later a few DSL customers. To other companies, we will provide 10 Mbit/s cable with RJ-45 [Ethernet]. We need some not expensive equipment which allows us to give each customer its own public IP (must control if client use only his own IP not other from our range). We need to control bandwidth speed for each customer, [provide] some stats, and it should have 8 to 24 ports. I guess that it needs to work in bridge mode so it won't use any additional IP space. Maybe somebody can give me an advice and links to some products.

[BB replied] "Anyone with a decent internet connection and one or more upstream ISPs willing to let you or even help you dabble with BGP routing can easily announce downstream "customer's" already owned IP address space. If it is a single cooperative ISP, they might even just do the BGP for your customer's own space for you, so neither of you even need an AS number.

Even with just your own space, whether directly from a registry, or allocated or assigned to you from an upstream ISP, you can easily slice and dice it to resell to friends and neighbors, but all subject to your contractual relationship with your upstreams. Some allow and even encourage such activity as it helps them sell you more bandwidth and keep you as a customer, and it is less hassle than dealing with smaller accounts they themselves may not really choose to do. OTOH, there are some LARGE titans / thugs out there that can be very ugly if they catch you "reselling" to the other half of your apartment building where they think THEY "own" all cable customers in it and expect to be paid by each user of their bandwidth.

If you are in an office park, or totally connected only by private property, or even just in range for any of various unlicensed radio or free space optical links, then you can probably do as you please. Or if you have local carriers that provide raw transport at attractive rates that you can use to profitably resell IP bandwidth, then you can have much farther reach. Otherwise, unless you are a common carrier yourself, your reach may be quite limited.

...

In any case, more information would be most helpful. Your hoping for bridging to eliminate waste of link address IP space probably indicates a quite small scale starting operation. I would note that point to point links, over perhaps a T1 or E1, traditionally use a /30, and still do on many networks. It is now quite "legal" (though not sure if "lesser-brands(tm)" routers yet support it) to use a /31 for point to point links. And, of course, you can run totally unnumbered—eg cisco IOS's: IP UNNUMBERED F0/0 type command that reuses another of the routers ports IP address—(eg the local ethernet IP address) also for its end of the WAN T1/E1.

If, perchance, you were simply reselling to folks in your office building, where you are within the 100 meter twisted pair range, and have a single common ground reference (the single main AC feed to the building with its local GROUND connection) so no additional protection (or better yet fiber) would be needed between you, you could take any even low end router that can do 802.1Q VLANS on its LAN interface, and stick each "customer" in its own VLAN. A good 24 port, even GIG-e capable on all ports (though I doubt you need that much...) smart switch capable of breaking out individual VLANs (and doing the tagging/untagging so your non-VLAN savvy customers just see untagged packets) onto individual ports—one per customer—is easily under $500 US, and probably even nearer half that.

Thus, with one physical router port at your end (but many virtual ones), you can give each customer a unique subnet of public REAL ADDRESSES, or NAT each customer's RFC-1918 internal space into an individual IP (to help identify the source of the inevitable infections "attacking" the rest of the internet), or, heck, NAT all customers into just ONE IP to save, save, save on IP space.

You can mix and match IP address options and control bandwidth used and also do whatever custom filtering is needed on a per customer basis. You can get a used cisco 2621 router on EBAY cheap enough. Do opt for the more pricey XM version if possible. Or any of plenty of similar boxes would work if you have a modest number of megabits of bandwidth. In the cisco world, look at "sub-interfaces"—each of which can be on a seperate VLAN and have whatever addresses and filters and rate controls you need. Really simple and is what every ISP uses for COLO customers, as well as what corporate users use for different departments or other internal groups.

Next probable step might be a cisco 7206 which has a vast array of different processor speeds and possible port options. You MUST know what you are doing or have someone you trust. An older non-VXR chassis will be very inexpensive on eBay, but won't take faster CPU and larger DRAM and some port options you may need.

...

You can also provide a local trunking service for customers with multiple rented spaces on widely scattered floors, that don't want to run their own cables throughout the building. This can be done between your switches, without your router being involved, at all. Probably from behind their firewall, that customer hands back to you a separate Ethernet you configure onto a another VLAN (not trunked to or known by your router) that appears at every switch of yours that feeds one of their scattered spaces within the building. If they use VLANs internally, your switches will need to be smart (also EXPENSIVE) enough to do Q in Q tagging.

You mentioned ADSL. Perhaps your office park is prelaced with POTS-only grade twisted pair for PBX stations? or you have a telco that takes an ATM connection (IMA'd T1/E1 or a faster T3/E3 or OC-x, etc ) as a feed to their ATM and FRAME RELAY and ADSL fed remote locations you can buy raw transport to?

Enough! I am rambling, and still am without any real idea of what you actually are trying to do.

[K answered] "We have our own IP range (from APNIC) and we are going (at least on the beginning) to connect companies from our building (max 20 companies each of them need have public IP). We offer webhosting, so we have a bit bigger pipe than the average company. I'm located in Taiwan so I need to deal with Taiwanese ISPs. For now, my company is fighting with them to allow us use our own AS number (which we got from APNIC and we need resign 1 month limit). Taiwanese ISPs are a mess. So forget about dealing with them in any normal way (not tomention prices). That is why we proposed to our neighbors to connect withus and avoid overpriced internet connections. It has benefits for us as well for them. We can afford an even bigger pipe, and they can have a faster and cheaper connection.

...

I thought about a few boxes and virtual routers for each company (using 4x pfSense per box with two interface cards), but I prefer something nicer. Or maybe one router an VLAN switch. Cable straight from switch to customer without any additional switches (maybe one or two).

...

I'm trying decide between price (overpriced Cisco) and easy to install and configure (few boxes with open source router)."

After several replies, including some nice prices from eBay, K explained that because he was in Taiwan, he could not follow the advice of the list and buy cheaply or on eBay.

"The situation of my company in Taiwan isn't funny. In our previous office, we got a 10 Mbps wireless connection.We decided to move to another office where our providers promised that we could have fiber and upgrade to higher speeds. After we moved, one of the providers said, sorry we cannot provide you internet there, you need find new provider. The second provider installed a pipe and route our new IP, but they said that it is impossible to peer. After a week they changed their mind and said that peering is possible but we need pay 4 x more per 1 Mbit/s.

...

So our current situation is 10Mbit/s one provider, VDSL modem with Rj-45 cable to our server room. All IPs are routed by provider. Target during next few months: 2 providers, 20-30 Mbit/s, BGP peering. Additional customers (up to 20) with 1 to 2 Mbit/s pipes. Equipment should be up to 60 Mbit/s, with redundancy. [We need to buy what we can find in Taiwan, in case we need a replacement box delivered quickly.]

— End