Sapphirehttps://blog.sapphire.moe
News and Development BlogMon, 03 Apr 2017 21:01:44 +0000en-UShourly1https://wordpress.org/?v=4.7.5March Update – Important Upcoming GitGud Changeshttps://blog.sapphire.moe/2017/march-update-important-upcoming-gitgud-changes/
https://blog.sapphire.moe/2017/march-update-important-upcoming-gitgud-changes/#respondMon, 03 Apr 2017 20:55:03 +0000https://blog.sapphire.moe/?p=242The end of March is here (and past) already and we’re posting and update post to catch you all up with what’s new and what’s about to happen. Upcoming April changes with GitGud We’re going to stop sign ups on GitGud.io starting April 15th. They will come back online with our new account system shortly … →

March Updates

We spent a lot of time during March trying to clean things up, reorganize, and working on our different areas of work for a major upcoming update release.

Sapphire

You may notice the domain we’re on for Sapphire is now changed. We wanted to have a more consistent name across, and so we changed the domain to match that consistency.

Sapphire is now incorporated as Sapphire Praxical LLC as of March 25th, 2017.

We’re planning on doing a soft launch of Sapphire Accounts around May 1st.

A post will be featured to show it off and talk about Sapphire Accounts more and how it will be a major dynamic for us.

Mixtape

Finances for Mixtape has been troubling. We aren’t getting enough donations in to meet our expenses. We’re running on minimum hardware needs and we can’t afford to drop another server.

We’re working on a new version of Mixtape to launch after Sapphire Accounts.

Tavrn

We’ve made major fundamental shifts in this project. Part of that shift was the branding and focus. The project, previous codenamed Spring, is now Tavrn.gg. Tavrn is a social network targeted for gamers, artists, and content creators.

We’re aiming for a Twitter like (microblogging) social experience first and combining that with features for people who play games, like to share their experiences, and allow devs to connect with players better.

We’ll announce more as things solidify. If you’re interested in trying the very early alpha of Tavrn, join our Discord.

TweetSave

TweetSave is having some backend problems. The backend is starting to not process requests fast enough. This may be related to the database size and the current hardware it is on.

]]>https://blog.sapphire.moe/2017/march-update-important-upcoming-gitgud-changes/feed/0February Updatehttps://blog.sapphire.moe/2017/february-update/
https://blog.sapphire.moe/2017/february-update/#respondTue, 21 Feb 2017 18:41:57 +0000https://blog.opengem.org/?p=218We’re approaching the end of February, and we’ve been fire fighting all throughout this month. We want to give you all an update on what’s been happening. Lost Hardware We lost Casper (mark 3) at the beginning of this month. The price had gone up (almost doubled) but we were unable to raise enough donations … →

]]>We’re approaching the end of February, and we’ve been fire fighting all throughout this month. We want to give you all an update on what’s been happening.

Lost Hardware

We lost Casper (mark 3) at the beginning of this month. The price had gone up (almost doubled) but we were unable to raise enough donations in time. We continually ran back ups while we were waiting for them to pull the plug. We migrated external projects like WrongThink and DesuArchive first on to our emergency back up server (Northern CA) temporarily donated by a Sapphire member. GitGud ended up in Southern California and Mixtape got put on Melchior our storage server for now.

Balthasar’s bandwidth had also exceeded 20TB/mo which meant we lost our 1Gbps uplink and were capped to 10Mbps for a bit. Track 4’s access was intermittent during this. We’ve since renegotiated our contract with the provider and now we’re back to full 1Gbps and we’ve completed copying all the data off of it.

Melchior running just Mixtape and TweetSave doesn’t have SSDs and it doesn’t enough disk IO, so it can lag sometimes. We’ll likely need to re-organize our tracks (data storage) on Mixtape soon, so we may have some maintenance windows coming up.

Cloudflare

We will likely have to abandon CloudFlare soon since we don’t have the funds for a Enterprise plan. This will definitely slow down everyone’s downloads and will kill any normal access to any other website hosted on the servers we use for download. So we must get the Mixtape front-end and TweetSave off our storage server as soon as possible.

New Hardware

Well enough for the bad news, the good news. After losing Casper, we did some soul searching. We need to plan for the future better. We need to find a way to cut costs without having a large investment up front. Should we run in emergency mode for a while to save up funds or should we look for a rent to own solution.

We found a NEW Casper (mark 4). It is an HP ProLiant DL180 G6 with dual X5650 2.66ghz CPUs, 24GB of DDR3 ram, and 6 x 24GB SSDs for $135/mo. This gives us a total of 12 physical cores and 24 logical cores. Best part is that will be ours after 12 payments. Meaning if we can survive another year we can keep the hardware and reduce our monthly cost.

While not as good as Casper mark 3 (we lost 4 cores, 104GB of ram, 3 SSDs, and about 30TB/mo) it is about a third of the cost and will help pave the way forward for us. We hope to rebuild and reestablish our projects back on this new Casper. This should help restore services levels closer to what they’re were before.

Project Spring

So recently, this happened; Michael got his Twitter account limited and the press picked it up and ran with it. Below is a list of all the media occurrences this showed up on.

The response we have gotten has been overwhelming, so thus we’re embarking on a slight detour, reviving an old social network project that we have had been working on for some time. We have brought on-board another developer to help build our vision (in Golang). We’ve been gathering feedback from various people and developers on Twitter and our Discord server. We hope to launch an early alpha soon in conjunction with our roll out of the first phase of Mixtape accounts on Opengem Accounts.

OpenGem accounts will be replacing GitGud’s authentication system and be providing a single-sign on for all our projects. After GitGud is up and running on OpenGem accounts then we’ll be able to launch paid Mixtape accounts using OpenGem accounts when accounts is ready. We may have to temporarily suspend GitGud sign up while we make this transition. We will likely lose 3rd party site sign on and 2FA (two factor authentication) temporarily until we can rebuild these for OpenGem accounts.

We’ll discuss more about Project Spring at a later date when we’ve solidified more information about it, the launch, the launch features, and what our vision is for the site.

Again thank you to everyone that has donated. We count on you now more than ever. If you’d like to Donate, visit: https://mixtape.moe/donate/

]]>https://blog.sapphire.moe/2017/february-update/feed/0Situation Update for Januaryhttps://blog.sapphire.moe/2017/situation-update-january/
https://blog.sapphire.moe/2017/situation-update-january/#commentsThu, 26 Jan 2017 21:49:43 +0000https://blog.opengem.org/?p=203We’re approaching the end of January, and we’re starting to have fires pop up. I want to detail some of the issues and obstacles we are now currently facing. Cloudflare Cloudflare is not very happy with us. They have recently contacted us about our bandwidth usage and asked us to get an Enterprise plan. We … →

]]>We’re approaching the end of January, and we’re starting to have fires pop up. I want to detail some of the issues and obstacles we are now currently facing.

Cloudflare

Cloudflare is not very happy with us. They have recently contacted us about our bandwidth usage and asked us to get an Enterprise plan. We use 350+ Terabytes of bandwidth per month for Mixtape.moe. On top of using a huge amount of bandwidth, Mixtape.moe, Cloudflare is very peeved by how 40% of our traffic and bandwidth goes to Asia, specifically South Korea. Asian bandwidth traffic is expensive.

The Enterprise plan for us would cost us $7000, and I’m not sure we can get to keep South Korea on that plan still. So now we’re left with the options of:

Get a $7000 Cloudflare Enterprise plan and possibly drop South Korea

Get a $300 Cloudflare plan and drop South Korea

Get dropped from Cloudflare by force

I don’t know about you guys, but those are some crappy options. Why would we want to abandon 30-40% of our user base? That’s insane. We’re not entirely sure how to handle this. Making our own CDN setup would be difficult. I’d like to get a dedicated server instance for Japan + South Korea to munch on.

Our Main Server

Our main server’s host is forcing our dedicated server plan to increase by $82 starting February 1st. The new server monthly cost will now be $359, up from $277/mo. This couldn’t have come at a worst time. We’re really struggling to pull along on donations each month. We had to add the new storage server, which was $99/mo more, and now we’re paying almost that extra now by force.

If we can’t afford the main server for new month, we’ll be forced to shift everything onto the main storage server. Services like GitGud may slow down a lot.

Previous Storage Server

We have a Hetzner server that served as the secondary storage server before the new storage server, and now is mainly a backup server. We’ve been transferring all of Mixtape off it to the new server, and it’s been taking a long time. Between the terabytes of files being moved and the secondary Mixtape track bandwidth, the server went over the 20 Terabyte limit from Hetzner and is now throttled to 10Mbps. We don’t have any extra money to pay for extra TBs. Track 3 has already been migrated, but Track 4 is still in the middle of transferring, and now it’s going to take a very long time to finish. We’ve setup Track 5 on the new server to start sending files to. But this means Track 4 files will be served from this 10Mbps server for the foreseeable future. Large files may become a nightmare to load.

Mixtape 2.0 and Accounts

We’re almost finishing up on the Account system. Currently we’re working on tying the Mixtape rewrite with that and getting the frontend completed. When that’s all done and ready to go, I’ll write an extensive blog post on the new changes, the redesign, and how the accounts work. I’m really excited about this, I just hope we get the opportunity to make it there before we hit a major financial wall.

We want to provide the best quality service that we can. Hopefully next month, we won’t have to sacrifice that quality to stay alive.

]]>https://blog.sapphire.moe/2017/situation-update-january/feed/28December 1st Updatehttps://blog.sapphire.moe/2016/december-1st-update/
https://blog.sapphire.moe/2016/december-1st-update/#commentsThu, 01 Dec 2016 19:20:28 +0000https://blog.opengem.org/?p=161It’s been awhile since we communicated, sorry about that! In between the last information post and now, we had a position seeking post that had some insights. It’s December 1st, and I want to update you all on what has happened, what is happening, and some stuff on what will happen. New Infrastructure We … →

]]>It’s been awhile since we communicated, sorry about that! In between the last information post and now, we had a position seeking post that had some insights. It’s December 1st, and I want to update you all on what has happened, what is happening, and some stuff on what will happen.

New Infrastructure

We got a new server!

We waited until black friday rolled around to try to get a great deal to fix a few major expansion issues:

We were running out of outgoing bandwidth on the 2 servers we already had, despite Cloudflare handling the majority of the bandwidth! At the time of writing this post, Cloudflare records us transmitting out 400TB of data in the last 30 days. Cloudflare handles 95.5% of the outgoing bandwidth. The rest is taken from our servers. Our servers also transfer files between each other for storage needs, and this is at a rate of about 1-2 terabytes per month of inter-server transfers, minimum.

We were running out of reliable storage. Our secondary storage server, before this new purchase, was not ideal in any form for storing files in. The storage drive configuration was not resilient to losing a drive and we had no real remote backup server. With the new server, we have a ton of space and can use the old storage server as a live backup for the foreseeable future.

We were at risk of losing everything if Cloudflare decided to kick us off their systems. Handling 400TB a month is no joke. The new server can handle 75% of that by itself. The rest of that can be made up buy our primary and backup server if in the case that we can no longer use Cloudflare’s excellent caching system.

The last point is of considerable concern. Last month, an image hosting site called Postimage.org went into a panic mode. [You can read about it here]. Postimage was using Cloudflare’s Pro plan for their CDN and to help alleviate bandwidth issues. Well it got to a point where Postimage.org was using way way more than Cloudflare liked.

Postimage.org posted this image of using 1.8 petabytes in the last 30 days on Cloudflare before Cloudflare shutdown their account.

Cloudflare doesn’t have an official limit of how much bandwidth an account or domain can use for any of their plans. But Cloudflare was extremely upset at Postimage.org for using 1.8 Petabytes of data. They demanded Postimage.org start paying $12,000 per month for using Cloudflare in this fashion, up from $20 per month. Obviously Postimage did not have this kind of money nor did they have the server infrastructure to handle this kind of issue.

So what does this have to do with us? Compared to Postimage.org, we use just 25% of what they use in bandwidth per month, which is still a crap ton! Like I said 400TB is not a joke, and that number is steadily climbing every week. We also only use a free Cloudflare plan because we can’t even afford to use an upgraded plan if we wanted to. So we were in a bad spot where at any time, Cloudflare could have audited our account and kicked us off their service and Mixtape.moe would have to shut down or move to a very limited server host that could support an unmetered but limited connection. This is mainly why we got the new server. We are prepared in the emergency case scenario that Cloudflare decides to send Mixtape’s account to the gulags.

Specifications of server

Intel Xeon E3-1230 V2 3.30 GHz (4 cores, 8 threads)

32GB RAM

4 x 8TB Hard Drives

300TB of outgoing Bandwidth

1 Gbps Uplink Port

The new server costs us $99 per month.

To explain a bit on how our setup will look like with this new server, I made a quick and pretty crappy diagram. There’s more happening than what is pictured, but this is the general idea.

New Plans and Directions

We’ve been planning and slowly doing work to progress Mixtape to have not only accounts, but premium paid accounts that offer additional capabilities for the user (and helps keeps up running and expanding!). We had a July 2016 user survey where people gave us valuable feedback on what they wanted to see happen and what they want to continue having on Mixtape.moe. We have to make decisions on how we want Mixtape to be in the future.

For the most part, I don’t anticipate the usage of Mixtape as it stands currently to change as we make these changes. My ultimate goal is to provide and improve the current offering and features of Mixtape while expanding beyond it. People are using Mixtape for a reason at this stage, and I don’t intend to abandon the users and use cases that jumpstarted Mixtape’s growth.

How we want to proceed on our plans for expanding Mixtape is slowly changing from our conception. Because these are not finalized plans and changes, I don’t want to make misconstrued statements, unfulfilled hopes, or give ideas that may cause concern. I would rather wait until the New Year or a bit afterwards to talk about these things. Below however I will talk about some stuff that is more immediate and decided.

New People

We reached out looking for a development partner to join. We’ve brought onboard a couple of people to help us develop Mixtape (and Sapphire as well) out in multiple areas. When things are finalized with their onboarding, we’ll make another blog post introducing them, their position, and specializations.

Mixtape

Mixtape.moe, as part of the first step of creating a new system, will be rewritten and based on Go (currently PHP and Pomf code base). The goal is to relaunch Mixtape.moe with at least the same functionality as it does now, fixed and improved API, and an account and login system that stores upload urls and the ability to delete your own uploads from within the account.

Mixtape is currently suffering some issues with the inhouse storage management system. Files on one of the storage containers show up as 404-Not Found. The files exist but are not being served. We’re trying to fix this as soon as we can.

Stats displayed are less than they actually are. We use a simple deduplication system and the stats use a simple file count, so the amount of files being uploaded is less than the reality.

GitGud

GitGud.io is just cruising along with no new issues or changes planned. GitGud is always updated to the latest GitLab version within 24 hours of a release.

GitGud still suffers from email confirmations going into Gmail spam and not being delivered at all on Mail.com addresses.

TweetSave

TweetSave.com is siting steady. The site will receive a frontend update in the future for visual improvements as well as giving it responsive layouts for mobile, tablet, and smaller width use cases.

TweetSave.com is gearing towards expanding to include extra functionality with premium paid accounts.

I’d like to give a big thanks to the people who donated to us, especially the people who help us out every month! We would be completely gone and shutdown if not for your support.

When it comes to finances, we are almost always just scratching by. Paid accounts will be critical for surviving 2017.

We have big plans, but we need to walk before we can run. I don’t wish to get ahead of myself on these plans and ideas before finalization and or the means to execute them. I hope to announce some of them early next year!

Statistics

]]>https://blog.sapphire.moe/2016/december-1st-update/feed/4Join Us! We’re looking for a development partner to take Mixtape to the next levelhttps://blog.sapphire.moe/2016/join-us-take-mixtape-to-the-next-level/
Sun, 02 Oct 2016 02:44:32 +0000https://blog.opengem.org/?p=129Status of this position: Closed We’re looking for development partner to help take our top 30k website (via Alexa) with 45.1 million unique visitors per month to the next level. What we’re looking for – Comfortable with small teams – Experience in scaling web applications for rapid expansion or server needs – Remote working, using … →

We’re looking for development partner to help take our top 30k website (via Alexa) with 45.1 million unique visitors per month to the next level.

What we’re looking for

– Comfortable with small teams
– Experience in scaling web applications for rapid expansion or server needs
– Remote working, using Slack/HipChat, Trello, etc
– Self-interest for the product being developed (File Hosting and Media Sharing service with a retro theme)
– Develop functionality of the web application front and back, though design work is not necessary, just front-end functionality
– A specific language does not matter, but we value speed and scalability

More about us and what to expect

We’re developing out the next level for our first and primary SaaS product, https://Mixtape.moe. We already have a large and rapidly growing user base. We’re in need of developing a new version of Mixtape.moe that funds itself through premium accounts with a fremium model based on what we have now for users.

We function as a very small and tight team. Mixtape isn’t the only service you’ll be limited to. Joining in on Sapphire, you’ll have the option to work on multiple projects if you’d like, or the ability to ask to join a team in developing their project. Mixtape isn’t the only product we plan to build out, but it is the primary project currently for income as well as the center of attention, eve for other projects. Other projects or products will have a direct impact on Mixtape.moe and it’s functionality in the future.

We have no intention in turning into a corporate structure of work and management beyond what is necessary for a professional atmosphere. We keep our front appearance and interactions with non-employees professional, but when working, being in your comfort zone is key so speak freely and willingly. Our goal as a startup is to bootstrap or self-fund ourselves entirely. We currently have no plans or desires for outside money to reach our goals. We want to make our own goals and decisions on our business, and that means keeping outside power from taking over. But this doesn’t come with it’s vast range of challenges and obstacles to overcome.

We prefer small teams for every project and will enforce a limit to prevent inefficient work and communication degradation.

We’re making products and projects entirely opensource. We’ll be developing open sourced web services that the community can get involved with and help check and fix bugs and security holes as well as show transparency in our systems. We value trust between the user and ourselves. Being open-sourced allows us to build trust, gain a community, and get support in finding and fixing issues in our software. It also means that users will depend on us to deliver them value in our online services.

Sapphire Values

1. Speed – This means everything. People want things to be fast and so do we. Every decision made for Mixtape and other projects involves on determining what a change will affect a product’s speed to the user. Loading times, uploads, downloads, responses from input, searches, everything needs to look, feel, and be fast. Having lean software is important.
2. KISS – Keep it Simple Stupid – Simplicity is important in how we work and what we make.
3. Politics and business don’t mix. Everything that Sapphire does is on a politically neutral or absent basis.
4. The long term picture – How we build things out now affects how it works later.
5. Empower the user and respect them – opinion of the users matter and they power us.

About Mixtape.moe

What started a small file sharing site has grown rapidly in just over a year to have 5+ million unique visitors, 1.9+ terabytes of files, 480,000+ uploaded files, and 325TB+ of bandwidth transmitted every month. We are growing at an alarming rate. Users love us because we allow a large upload limit for free, protect users from false abuse flagging, provide the ability to share any kind of file, and of course a simple and fast loading homepage for easy uploading and fast download speeds.

Mixtape follows a modernized retro theme and uses audio/tape/video terminologies as a fun branding mechanic. We are mainly interested in targeting consumers with the Mixtape brand. We want to build out Mixtape to be a hybrid of a media sharing site and a file hosting site. Building a community around shared media files like videos, images, and audio as well as give the users the ability to also share any kind of file they wish in the same place.

Challenges

Mixtape has a great deal of challenges to face as a file hosting site:

Bandwidth – We use an extraordinary amount of bandwidth and that’s only going to grow exponentially. We currently use over 320 terabytes per month at our size.

Storage – We use a lot of storage space as a file host. But the storage space isn’t completely unmanageable, for now. We handle almost 2 terabytes of files total right now.

Malware – Users will try to upload or share files (or even unknowingly share these files). We keep Mixtape as clean as we can manage. It is important to protect end users from malicious attacks as well as protect Mixtape from being flagged as a malicious website on blacklists. Part of this difficult revolves around scanning the entirety of Mixtape’s uploaded files in an appropriate amount of time.

Speed – We always keep speed a priority. Important to keep the experience seamless and load instantly for the user. Responsiveness is important. Handling how to ensure the site stays as responsive as possible with so many users as well as handling uploads and background processes is important to keep in mind on all decisions.

Integrity of URLs – Once someone has a Mixtape URL, that URL has to always work no matter what.

]]>#125 – You can’t make a deal if you’re deadhttps://blog.sapphire.moe/2016/you-cant-make-a-deal-if-youre-dead/
https://blog.sapphire.moe/2016/you-cant-make-a-deal-if-youre-dead/#respondMon, 11 Jul 2016 07:30:51 +0000https://blog.opengem.org/?p=79Rule of Acquisition #125: You can’t make a deal if you’re dead. To be a donations based organization. You need to be able to accept donations. So we had to put together a campaign to raise funding. Putting together a crowdfunding campaign isn’t easy. Who are we? Why should people give us money? After a … →

To be a donations based organization. You need to be able to accept donations. So we had to put together a campaign to raise funding.

Putting together a crowdfunding campaign isn’t easy. Who are we? Why should people give us money? After a lot of thought and brain storming we finally started realized by looking at what we already had done, that it showed a trajectory that we might be able to guess what we might do in the future from. While we could just be a collection of personal projects, we realized there were some common threads that ran through our projects. We believed in free speech, archival, ease of use, good tools, good design, far reaching goals, and open source software. We used this to form our identity as a group. When people contribute to us, they’re going to be contributing to these ideals.

We created a campaign that put our projects on display through this lens. Everything is tied together through our group’s mission. Then we had to figure out a target goal; what was reasonable?

With my past experience, I realized that I could fundraise about $2,000 myself but Michael had much better success in the past with donations and had a much larger social reach. So we aimed at little under $5,000. If we went over $5,000, that could be rounded up to $10,000 in people’s eyes, if you’re not careful. (see the numerous articles about the Freakonomics/psychology behind .99 cent pricing).

We were already paying over $200/mo for everything, but we knew in about a year that we were likely to double in size at our current growth rate. So a budget of $400/mo. would make this money last roughly a year which was a nice roundish number.

We also had to choose how we were going to fundraise, raise or lose it, keep what you raise in a fixed amount of time, or raise as you go. For what we were doing, the raise as we go made the most sense. We didn’t need a lot of capital upfront to buy hardware, we just needed to pay our hosting bill each money. GoFundMe provided the best set up for this.

We decided to also launch our campaign 2 months before we needed the money, in order to give the campaign time to raise enough money.

We launched the campaign October 22nd and within 30 days had raised 10% of our goal. It was successful enough to see a light at the end of the tunnel. We might be able to continue doing what we love. Our users found enough value in what we were doing to support us. We’re so appreciated of this, I don’t know how to express it.

So now that we had some funding, by launching before 2016, it also lined up with another advantageous event for us. We really wanted to make sure we spent our funds wisely because we’re not sure the campaign would continue to fund at the same initial rate. And we want to be the best stewards of our users’ money as possible and Black Friday was right around the corner.

We used our thanksgiving break to scour all the providers looking for the best deal we could. We created spreadsheets, weighing the pros and cons of cpu, memory and disks config.

We had SSDs with our previous provider. One of the continuing goals of Mixtape is to be the fastest Pomf clone, so we really wanted to continue to have SSDs, however if we sacrificed performance for storage, we could use old slow platters (hard drives) get much more space. Other factors like RAID configuration played into the speed/space mixed. So even if we went with slower disks we might make up for it by using multiple hard drives.

We also debated about location, since we had to consolidated a bunch of VPSes, we’d need one big server and we wanted it to be in the central part of the U.S. so we had good quick routes to Asia and Europe.

So when we saw a dual 2.4ghz octo-core with hyperthreading with 128gb of ram for a steal. So that’s 16 physical cores and 32 logical threads about 2-4 times more than we currently had and about 4 times the memory for the same price. And the disk configuration was just something we could not beat, 9 SSDs drives. Not only were the drives super fast, we’d be able to use RAID to increase the speed further. And with 9 drives, it didn’t matter how small they were, it was still was 10 times more than we had. A find like this quickly draw a close to any discussion or arguments we had over various options.

We ordered it and drew up plans to migrate off our VPSes. Though we did chose to hang on to one of our older servers in the EU because we did have a good deal on it and it had enough space to hold back ups for this new main server.

Through planning and hard work, we continually challenge each other to do better. We listened to our users and community and continue to deliver the best value we can with our resources. This is going to be a requirement if we want to hit those long term overarching goals in the future.

]]>https://blog.sapphire.moe/2016/you-cant-make-a-deal-if-youre-dead/feed/0Rule of Acquisition #3: Never spend more for an acquisition than you have to.https://blog.sapphire.moe/2016/hosting-never-spend-more/
https://blog.sapphire.moe/2016/hosting-never-spend-more/#respondSun, 03 Jul 2016 12:00:17 +0000https://blog.opengem.org/?p=65One of the things that got me interested in Sapphire was Michael’s openness about talking about his infrastructure and how much he was spending to host his services. Here’s the layout of the servers soon.http://t.co/wERmVLyqou will get it’s own server and a euro 8/v/ game server. pic.twitter.com/x2vRAxJb4y — Drybones △ (@Drybones5) April 8, 2015 Reading … →

Looking at those providers, their prices were impressive. How much value do they actually deliver. I have several dedicated and collocated servers myself. And I had never paid less than $50/mo for a server, and that server was horrible. I realized that networking with other system administrators would not only reduce my server bill but may help others as well. I have a list of my own research and tricks that have allowed me to move mountains with very little (and plenty of failed mountain moving to know what not to do)

So after talking with Michael and he was showing me all these challenges he was facing from Mixtape, GitGud, and 8archive. I realize he was a fellow mountain mover but our skills complimented each other well. Plus he was really polite and easy to work with. You can tell he really listens and you are heard. I joined Sapphire.

One of the previous start ups I had worked with, I had to design an online long-term archival system. But how do you charge for lifetime hosting? When the server bill shows up every month how can you provide safe secure fast hosting for an easy fee structure that’s easy to understand. It’s quite a challenge. We ended up tying it to user’s lifespan, so it not really forever. So younger users would cost us more than older users but our primary demographic was largely users over 50, so it minimized our risk.

So when I saw “Pomf” clones that provided unlimited storage indefinitely, I quickly realized these aren’t your standard hobby sites but ticking time bombs. And Sapphire’s Mixtape file host was no exception. Sapphire was running on a bunch of VPSes and the monthly cost was rising quickly as we need more and more resources to continue operations.

Now there’s “shared” hosting where you’re given user level access and a fair amount of resources that you share with a bunch of other users. Then there’s “cloud” or “VPS” hosting, where you’re given root (the highest level access) to amount of resources that you share with a bunch of other users. Then there’s “dedicated” or “bare metal” servers, this you’re given root but all those resources are 100% yours.

On all the previously described hosting levels, if any hardware fails the hosting company will (Usually, not all) replace it free of charge. So you rent and get some basic insurance. The next level after “dedicated” is “colocation” where you buy your own hardware and collocated in it a data center and only pay for space, power/cooling, and bandwidth. However this requires much more upfront investment to buy the server hardware upfront and if anything goes wrong or dies, you’re on the hook for the replacement. And if you have to buy online and ship the part, this can add days of down time.

How to decide what you need and don’t need? Well do you need root level control? VPS and shared hosting is generally good for any website starting out. But you need to be aware when you’ve outgrown your VPS. If you’ve had to upgrade and are paying $50/mo, you could be getting a dedicated server for that much.

Sapphire is completely run out of Michael’s pocket and user donations. To survive we’re going to need to be very frugal with our spending and make good long term decisions. So far so good. We continue to scour the web and locate deals on resources and figure out how we might be able to work them into our infrastructure. We have tried out several value providers and are learning the ins and outs.

]]>https://blog.sapphire.moe/2016/hosting-never-spend-more/feed/0Rule of Acquisition #95: Expand or Diehttps://blog.sapphire.moe/2016/rule-of-acquisition-95-expand-or-die/
https://blog.sapphire.moe/2016/rule-of-acquisition-95-expand-or-die/#respondSun, 12 Jun 2016 02:46:25 +0000https://blog.opengem.org/?p=28It’s been an interesting year so far to say the least. Things are looking a bit different than I imagined. I want to talk about a few things in this post: Mixtape.moe’s Unexpected Surge in Use Mixtape.moe’s Future and Expansion Sapphire Accounts + Mixtape Premium First Mixtape’s growth. On July 9th, 2015, I opened up Mixtape.moe running … →

]]>It’s been an interesting year so far to say the least. Things are looking a bit different than I imagined.

I want to talk about a few things in this post:

Mixtape.moe’s Unexpected Surge in Use

Mixtape.moe’s Future and Expansion

Sapphire Accounts + Mixtape Premium

First Mixtape’s growth.

On July 9th, 2015, I opened up Mixtape.moe running on the simple Pomf PHP script. I had no idea how far it would go at the time. In the first week and we had already hit 100GB of bandwidth usage in a week.

Mixtape.moe’s bandwidth during the first week of operation. 100GB of bandwidth.

Key elements I wanted to hit was to be very fast. Instant homepage load to quickly upload something. Fast upload and URL generation. Fast access times to view a file.

Kept a minimal homepage design, very little code to load, compressed and cached elements for what little was needed. I spent a lot of time pouring over NGINX and PHP5 documentation and guides to optimize the site’s configuration and request times to be as small as possible.

Fast page loads and a simple frontpage

I even hosted the site on Solid State Drives too. I didn’t want anything to slow the site. No I/O wait times and fast read access times. I don’t have a RAID of 50 HDDs. I had the option of at best a RAID 1 of consumer drives or some SSD VPS on Linode. I went with the VPS. Linode had a reliable network, 40Gbps In and 1Gbps out. $80/mo for 192GB SSD VPS.

Close to the original look of Mixtape.moe. Taken November 16th, 2015.

Approaching December, we got ourselves a new dedicated server. And to make things fresh, I made a change to Mixtape to be more mobile friendly, responsive, and cleaner. More space to drag and drop.

We also hit 200 Gigabytes of files and 60,000 files total. We needed the new server for space and other sites we were managing in a consolidation attempt.

Thanks to donations, we were able to expand here and survive. And apparently thrive. Thank you, to all who donated to us.

Third Design change of Mixtape.moe.

We hit 20 Terabytes per Month on bandwidth usage, December 10th, 2015

December 11th, 2015. 20TB per month

Then 60 Terabytes per month.

April 10th, 60TB bandwidth per month

And now today Mixtape.moe’s statistics look like this as of this post:

130TB of bandwidth transferred per month

3,000,000 unique visitors per month

1.23TB of files uploaded

330,000 files uploaded

50,000 files uploaded per month

Average of 1,500 – 3,000 uploads per day

65 – 120 uploads per hour

Alexa ranking: top 55,000 globally, top 18,000 in the United States

Traffic:

65% – Webm + MP4

12% – GIF

10% – JPG + PNG

13% – Other file types

130TB per month

Mixtape 3 million visitors

Alexa ranking – 55,661 on June 10th, 2016

No kidding

Mixtape on June 10th, 2016

It’s not even the 1st Anniversary of Mixtape.moe and we’re hitting new growth issues we didn’t expect to hit for another year at least. We didn’t anticipate such a surge in usage. We don’t even have any new features. We’re still very rudimentary in how our site functions.

Rule of Acquisition #95: Expand or Die

Splitting this next part about Mixtape’s future into two parts: Hardware and Software

Our new dedicated server has enough processor power and memory to handle Mixtape. But Mixtape has different resource needs. It needs:

A TON of bandwidth

A TON of storage

Fast storage access

A minimum of 1Gbps Up and Down links

We’ve already maxed out our new server on bandwidth and storage use. We’re not in an immediate danger as we have another server to cruise on until we get new resources. But we’re not a wealthy group. We’re meticulously budgeting our next move. We’re not replacing this dedicated server, we want to add on to it. We’re looking to purchase ourselves a 4U rack mount server to put into a data center via colocation hosting. It’s more upfront cost but it’s a more sustainable monthly cost for us to handle. It will pay for itself within a few months in action.

We want to get a very large storage node going to store our Mixtape files in. Talking up to 8 to 24 drives in RAID. 48TB – 96TB with 3.5″ HDDs or 8TB – 24TB with 2.5″ SSDs. We’re bias to the 2.5″ slots to fit more slots and use SSDs. We’ve been very happy with how SSDs have been working out for us. The speed and seek times are great and very useful for malware scans, mass modifying of files, and getting meta data information on files fast. But we’ll take what we can find and can afford. I’ve very excited about expanding. Kind of giddy to get working on another server.

Let’s talk software changes.

We’re going to be making a wide variety of changes to Mixtape. Our #1 priority of Mixtape is to NOT change how it works for public uploads.

Mixtape’s public uploads will always be free

Up to 100MB uploads

Not require an account

Will allow direct raw file access

With that said, we’re going to expand on the functionality of Mixtape.

We’re planning on adding file preview/view pages for media. This allows social media sites to be able to grab metadata and meta media to show preview cards and media files on posts. For example the following tweet below contains a link to a Youtube video. We want to have Webm and MP4 video files embed just like this and also allow embedding of GIF, JPG, and PNG image files as well when linked.

On top of post preview cards, it allows people to upload Webm files that have the option of looping, amongst other possible options.

Part of the premise of “Expand or Die” is that Mixtape needs to become more self-sufficient. We rely on donations ENTIRELY. Donations are not guaranteed and at some point we won’t be able to make up the costs of a large network of servers with just donations. So we’re working on Sapphire Accounts.

A Sapphire Account will be tied to multiple services provided by Sapphire. The first and most major will be Mixtape. Mixtape’s premium accounts will feature more account options, much greater upload sizes per file, and a large pool of storage attached to your account. Accounts will have a free tier that add the functionality to keep track of your upload history and URLs and delete them later.

We’re don’t have more detailed information to provide for now. A feature preview in a future blog post will showcase the changes and another post will go in depth on the Mixtape account upgrades.

Again, thank you to the donors who help keep us up. Every we do is for them and we hope to give back to you all with a solid service.