tag:blogger.com,1999:blog-6831226987864085895.post6172128824980952526..comments2015-04-29T16:53:26.757-07:00Comments on Extreme Media Servers: Wiring up the PMs for PowerDon Honabachhttp://www.blogger.com/profile/10277680683125440529noreply@blogger.comBlogger27125tag:blogger.com,1999:blog-6831226987864085895.post-15727852242220762572012-09-17T10:00:13.735-07:002012-09-17T10:00:13.735-07:00If you want to buy all the components, use this si...If you want to buy all the components, use this site: www.ybytes.euAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-64977357333870954722010-10-18T16:54:50.573-07:002010-10-18T16:54:50.573-07:00Additionally, most PSUs have a combined wattage li...Additionally, most PSUs have a combined wattage limit on 3.3v and 5v that you have to be careful not to hit.<br /><br />As an example, I&#39;ve got a 500W PSU up on newegg that I was looking at. Looking at the sticker, it can provide 20A on 5v and 24A on 3.3v, but 5v and 3.3v can only provide 120W max. If the 5V rail is pulling all 20A, that only leaves about 6A for the 3.3v, and conversely, if the 3.3v line is using all 24A, that only leaves about 8A for the 5v line.<br /><br />Additionally, the combination of 3.3 and 5 might also share a limit with a third voltage line before the combined limit of the PSU itself.coderjoehttp://coderjoe.livejournal.com/noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-62887364096693102152010-10-18T08:26:22.754-07:002010-10-18T08:26:22.754-07:00@Yeroc
In an ideal world...@Yeroc<br />In an ideal world...Cockroachhttps://www.blogger.com/profile/17602140710202536819noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-62030024390768301212010-10-18T08:24:39.188-07:002010-10-18T08:24:39.188-07:00It seems to me the proper solution is to find a co...It seems to me the proper solution is to find a controller that supports staggered spin-up of the devices so you don&#39;t need to over-build your PSU just for the bootup scenario.Yerocnoreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-48036285974656166622010-10-17T06:56:12.291-07:002010-10-17T06:56:12.291-07:00I have 30A@5V on my PSU but it&#39;s &quot;only&qu...I have 30A@5V on my PSU but it&#39;s &quot;only&quot; 80A@12V - so it couldn&#39;t spin-up 45 drives and the system anyway. I&#39;ll be using a second PSU when I have enough drives.Cockroachhttps://www.blogger.com/profile/17602140710202536819noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-89404391740660966452010-10-17T03:47:00.868-07:002010-10-17T03:47:00.868-07:00I should clarify.. I haven&#39;t seen PSUs that ca...I should clarify.. I haven&#39;t seen PSUs that can supply enough on the 5V rail to satisfy that 22.64A in addition to the draw from the rest of the system: fans, mobo/cpu/ram/cards, plus the port multipliers themselves.coderjoehttp://coderjoe.livejournal.com/noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-30208421731709433752010-10-17T03:26:02.837-07:002010-10-17T03:26:02.837-07:00@Cockroach:
To supply more current on the 5v rail...@Cockroach:<br /><br />To supply more current on the 5v rail? Modern PSUs supply a lot more current to the 12v rail(s) than the 5v rail. Adding the circuit I mentioned provides more 5v current.<br /><br />45 of the WD green drives would require a maximum current of 22.64A from the 5V rail, if all are reading or writing at once, which I suspect is going to be fairly likely at boot. I don&#39;t think I have seen a PSU with that kind of 5V rating.coderjoehttp://coderjoe.livejournal.com/noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-772255388070870782010-10-17T01:32:56.290-07:002010-10-17T01:32:56.290-07:00Why would you need to use any electronics rather t...Why would you need to use any electronics rather than just connect it all together?Cockroachhttps://www.blogger.com/profile/17602140710202536819noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-14748719638557357822010-10-17T00:01:16.830-07:002010-10-17T00:01:16.830-07:00I realize this is a necropost, but more on the pow...I realize this is a necropost, but more on the power consumption. I just looked at the WD data for their green drives.<br /><br />spinup: 1.75A @ 12V<br />read/write: 296mA@12v, 503mA@5v<br />idle: 243mA@12v, 172mA@5v<br />standby/sleep: 3mA@12v, 160mA@5v<br /><br />Again, spinup is a killer.<br /><br />With some electronics knowledge, it would be possible to make some 12V pcie connector to 12/5 4-pin power connector boards. An LM7505, some transistors to add more current capacity (the LM7505 usually only can provide up to 1A by itself), and a few other parts, really.coderjoehttp://coderjoe.livejournal.com/noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-33430861511715491702010-05-19T15:03:56.601-07:002010-05-19T15:03:56.601-07:00&gt;&gt; Do I need to just add another power suppl...&gt;&gt; Do I need to just add another power supply?<br /><br />The short answer is YES! If you are going to have a live set of 45 drives, just get a second 400/500 Watt PS for the MB, Boot HD, and Fans. It is a nice way to segment things out as well as the system seems to work better when you power up the 45 drives first and then fire up the motherboard.<br /><br />I&#39;ve gotten away with a single PS and 45 live drives, but if all drives are going, the system was unstable which is no fun.<br /><br />DonDon Honabachhttps://www.blogger.com/profile/10277680683125440529noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-28164120916964353842010-05-19T08:53:19.889-07:002010-05-19T08:53:19.889-07:00Hello,
I&#39;m in the process of ordering all of ...Hello,<br /><br />I&#39;m in the process of ordering all of my parts. I&#39;ve already ordered the 1200W power supply, having not yet read all of these insightful comments.<br /><br />Did anyone come up with a firm resolution to this potential disaster or am a screwed?<br /><br />I will most certainly be using all 45 drives and they won&#39;t usually be sleeping or idling. I haven&#39;t been able to find a 12v-to-5v &quot;computer&quot; converter, most everything seems to be both large and automotive in nature. Do I need to just add another power supply?<br /><br />Any help would be appreciated.Heywardhttps://www.blogger.com/profile/00960991928254274237noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-28926404566706057042009-11-19T16:29:48.665-07:002009-11-19T16:29:48.665-07:00Hmm... decided to check out the consumption of the...Hmm... decided to check out the consumption of the non-LP 1.5TB barracuda (ST31500341AS)<br /><br />Spinup: 2.8A @ 12V<br />Idle: 0.330A @ 5V, 0.530A @ 12V<br />Operating: 0.700A @ 5V, 0.610A @ 12V (40% r/w, 40% seek, 20% inop.)<br />Seeking: 0.430A @ 5V, 0.663A @ 12V (random, 20% idle)<br />Standby/Sleep: 0.130A @ 5V, 0.021A @ 12V<br /><br />so, while operating, it appears that this drive uses a bit more power on 5V than 12V. Another thing I just noticed is that the measurements were taken with an ambient temperature of 25 degrees Celsius, so as things get warmer, power usage might change.coderjoehttp://coderjoe.livejournal.com/noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-69002376542700351582009-11-18T03:33:27.589-07:002009-11-18T03:33:27.589-07:00WD doesn&#39;t have much info in their datasheets ...WD doesn&#39;t have much info in their datasheets anymore, as far as power consumption on the different voltage lines, but Seagate&#39;s datasheet for their Barracuda LP series claims that the following:<br /><br />For the 1TB model:<br />spinup = 2.0A @ 12V<br />Idle = 0.218A @ 5V, 0.216A @ 12V<br />Operating = 0.335A @ 5V, 0.350A @ 12V<br />Standby and sleep = 0.225A @ 5V, 0.096A @ 12V<br /><br />For the 1.5TB and 2TB models:<br />Spinup = 2.8A @ 12V<br />Idle = 0.168A @ 5V, 0.337A @ 12V<br />Operating = 0.360A @ 5V, 0.420A @ 12V<br />Standby and sleep = 0.350A @ 5V, 0.010A @ 12V<br /><br />These figures go against what backblaze said about most of the power draw being on the 5V lines.coderjoehttp://coderjoe.livejournal.com/noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-50799619910454671002009-11-18T02:31:48.060-07:002009-11-18T02:31:48.060-07:00That definitely sucks... Detecting a problem and h...That definitely sucks... Detecting a problem and having an answer like that: &quot;We won&#39;t fix it anytime soon.&quot; :(Carlos Martinshttps://www.blogger.com/profile/06977756139964818233noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-77510490219265421812009-11-17T21:57:58.836-07:002009-11-17T21:57:58.836-07:00Don
@reverendjw... Thanks for the script you post...Don<br /><br />@reverendjw... Thanks for the script you posted on the blog - awesome! Will be giving that a try this weekend! <br /><br />As for the low noise fans - I just found some we had in our shop and went with those. <br /><br />As for noise, right now with 42 drives in the box (majority of them in sleep mode), it is putting out between 55 to 62 DBs accordingly to a Radio Shack Sound Meter that I bought years ago and never used until now ;)<br /><br />For what it&#39;s worth, I hadn&#39;t noticed the noise frankly until I read the noise comments on the blog. However, definitely wouldn&#39;t want it in a media room though as is.<br /><br />I could definitely let the fans go slower than they currently are and reduce the noise with my setup that has most of the drives in sleep mode. Just not a lot of heat going on.<br /><br />Ultimately I think what we need is a multiple fan temperature based controller to keep the noise as low as possible. If anyone has a link to one that can support a min. of 5 fans and preferable 6, please post.<br /><br />I haven&#39;t shock mounted my drives yet. I know this is &quot;bad&quot;, but with my non-RAID type setup and only a few drives being awake at the same time, there just isn&#39;t a heck of lock of vibration going on. In short, its on my to-do list, just not at the top right now.<br /><br />I&#39;m also working on a post about an issue that I submitted to Microsoft about accessing junction points under a Windows Share remotely - the junction points (i.e. hard drives) unfortunately just disappear from the list in Explorer but can still be accessed by typing in the full path).<br /><br />Not the end of the world and there are workarounds just kind of a pain. With any luck Microsoft will release a fix in their next Service Pack round, but I wasn&#39;t lucky enough or big enough to justify getting a hotfix made for it. It was nice though hearing that they duplicated the issue internally. It was then sad to essentially hear that they wouldn&#39;t be fixing it any time soon! If anyone works at Microsoft and can nudge this up the line, here&#39;s the case # (REG:209100574446845002).<br /><br />I also ordered in a set of 15-2GB Drives to replace my 30-1GB units. When I get those and replace them, I&#39;ll do a post on the actual 5V and 12V rail usage for each Port Multiplier. This should help those that want to try and stick with a single power supply or just want the field #s.<br /><br />DonDon Honabachhttps://www.blogger.com/profile/10277680683125440529noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-64088436804796515652009-11-17T21:11:50.531-07:002009-11-17T21:11:50.531-07:00@Don
There&#39;s been a few comments about using ...@Don<br /><br />There&#39;s been a few comments about using low noise fans...a definite must for keeping one of these things in the house, or if you have an apartment with no garage to keep it in ;-)<br /><br />I was curious, have you measured the actual sound output from your pod?<br /><br />also, what about something simple like shock mounting the drives or wrapping them to reduce vibration?reverendjwhttps://www.blogger.com/profile/11149370645108059893noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-38858132834018834052009-10-28T02:54:12.923-07:002009-10-28T02:54:12.923-07:00All the &quot;multi-rail&quot; issues were part of...All the &quot;multi-rail&quot; issues were part of the reason why I chose a single-rail PSU for my PC. The <a href="http://translate.google.com/translate?client=tmpg&amp;hl=en&amp;u=http%3A%2F%2Fabertoatedemadrugada.com%2F2009%2F09%2Fcorsair-hx-850w-psu.html&amp;langpair=pt%7Cen" rel="nofollow">Corsair HX850W</a>. :)Carlos Martinshttps://www.blogger.com/profile/06977756139964818233noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-85916178813140696022009-10-27T16:58:42.579-07:002009-10-27T16:58:42.579-07:00Possible power supply changes...
&gt;&gt; Seagate...Possible power supply changes...<br /><br />&gt;&gt; Seagate ST31500341AS for example uses 0.7A on 5V (operating) which totals 31.5A with 45 drives. This is more than your PSU is rated, and there will be other things going on too.<br /><br />I finally got a chance to do some real world testing on my WD Green Drives and surprisingly they pull a significant amount of power off the 5V line despite the data I got from WD on this (about the same as the Seagates). This combined with the PM backplane would definitely kill the 5V RAIL Power Budget for the 1200W Power Supply.<br /><br />I still need to do more testing/verification as the #s I got from WD Tier 2 support indicate that the power budget should be about 1/5th of the numbers I&#39;m seeing so I want to make sure my data is correct.<br /><br />However, at this point I have two options assuming the data is correct or you want to run more power hungry drives:<br /><br />1. Stick with a single power supply and find a 12V to 5V power adapter and switch probably 3 of the PMs over to it. This will use one of the 12V rails from 1200W PS and on initial check I appear to have enough left on those RAILs/power budget for this.<br /><br />2. Go with the more &quot;traditional&quot; BackBlaze Dual Power Supply arrangement.<br /><br />I&#39;ll update as I get a chance to do more verification, but for now, just be aware if you go with a single power supply that you may run into issues.<br /><br />Also, be aware that I&#39;m currently running 42 drives without ANY problems even though I&#39;m over my power budget. However, I use power management so the majority of drives are asleep most of the time so the voltage overload isn&#39;t an issue and the power supply seems to handle the initial surge overload on start up just fine. <br /><br />DonDon Honabachhttps://www.blogger.com/profile/10277680683125440529noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-54630104977412936682009-10-27T16:47:32.724-07:002009-10-27T16:47:32.724-07:00Possible power supply changes...Possible power supply changes...Don Honabachhttps://www.blogger.com/profile/10277680683125440529noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-53740817736077506832009-10-22T13:28:24.882-07:002009-10-22T13:28:24.882-07:00Yeroc,
&gt;&gt; Are you using a controller/bios/d...Yeroc,<br /><br />&gt;&gt; Are you using a controller/bios/driver combination that supports staggered spin-up?<br /><br />As far as I can tell, the PM boards nor the SATA SilImage Based Cards support this option.<br /><br />On some WD drives, there is an option for what appears to be standby mode on power up, but I couldn&#39;t find a way to get the drives to ever spin up after power was applied when using this jumper. Maybe there is an expert out there that nows of a low-level command/API for this???<br /><br />My testing indicated that I draw about 900 Watts during initial spin-up with ~40 drives on the system. With the 1200 Watt power supply, that draw appears acceptable with room to spare.<br /><br />While it is working without staggered spin-up now, I would have to have a staggered spin-up option. My gut tells me with the existing H/W it isn&#39;t possible though.<br /><br />One of the ideas we&#39;ve tossed around is a power relay board that when powered is applied would stagger the spin up of each PM/cluster of 5 drives. If we go this route, I&#39;d probably add a USB interface to the power relay board and a way to control it via software so you could power down PMs without shutting down the server. <br /><br />Also, FYI, if you take a look at the prior post, there are pictures of the wattage used for each stage of operation: spin-up/power on, etc.<br /><br />DonDon Honabachhttps://www.blogger.com/profile/10277680683125440529noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-18962840273820012672009-10-22T13:12:06.975-07:002009-10-22T13:12:06.975-07:00Thanks for the interesting articles.
You don&#39;...Thanks for the interesting articles.<br /><br />You don&#39;t mention using staggered spin-up to reduce the peak power requirements at startup. Are you using a controller/bios/driver combination that supports staggered spin-up?Yerochttps://www.blogger.com/profile/00665221564150100159noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-60213429504477672412009-10-22T08:21:39.029-07:002009-10-22T08:21:39.029-07:00Keep it coming Don, this is great stuff.Keep it coming Don, this is great stuff.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-19966583879522920102009-10-19T09:51:44.300-07:002009-10-19T09:51:44.300-07:00Hey Carlos -
The wholesaler that got me the PMs ...Hey Carlos - <br /><br />The wholesaler that got me the PMs asked if I thought there would be any interest for a 30 bay &quot;mini&quot; version.<br /><br />I think he is looking at something like a 100+, but if you are interested, it might be worth dropping the company a note.<br /><br />http://chyangfun.com/<br /><br />DonDon Honabachhttps://www.blogger.com/profile/10277680683125440529noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-56639856108316468152009-10-19T09:49:40.510-07:002009-10-19T09:49:40.510-07:00&gt;&gt; Have you tried this with all 45 drives?
...&gt;&gt; Have you tried this with all 45 drives?<br /><br />I currently have 42 drives in the EMS and haven&#39;t had any issues (knock on wood!).<br /><br />I am using the WD Green 1TB and 1.5TB Drives though for the majority of them.<br /><br />Interesting point on the 5V draw. I looked at WD&#39;s site and couldn&#39;t get the numbers on the 5V versus 12V line though. <br /><br />Worse case, I&#39;ll have to put in some 12V to 5V converters to move the load around on the rails.<br /><br />For what it&#39;s worth, in my setup, I won&#39;t be accessing all the drives concurrently. In the typical usage scenario, 39 to 41 of the current 42 drives will be asleep/standby mode. I can&#39;t think of a single instance where I would have all 42 (or 45 for that matter) drives actively reading/writing data for my type of usage.<br /><br />With that said, now that I know it might be an issue, I&#39;ll be pulling out the AMP meter and see if I need to re-rig some of the 5V power lines for my drives.<br /><br />This brings up a side note, with this many drives, you definitely want to stay away for the uber performance, uber power eating drives. Not only do they generate more heat, but they&#39;ll make life much more difficult in managing your power budget as you get the case fully loaded. <br /><br />Thanks,<br />DonDon Honabachhttps://www.blogger.com/profile/10277680683125440529noreply@blogger.comtag:blogger.com,1999:blog-6831226987864085895.post-70779904007281897702009-10-19T05:50:08.355-07:002009-10-19T05:50:08.355-07:00Thank you for sharing this with us! You gave some ...Thank you for sharing this with us! You gave some very good information!Anonymousnoreply@blogger.com