¿ªÔÆÌåÓý

OT - NAS Drives?


Jim Waters
 

I am getting tired of USB3 SSD's and HDD's floating around that I use to store my PI Projects.? Its no problem for me to build a NAS but I don't have the time or parts.? I have no experience with off-the-shelf NAS Drives.? What are good manufacturers and models to start looking at?? I am currently checking out Western Digital and?Asustor.? What do other do to store images?

I would need RAID 1 (minimum) and about 6 to 8TBytes of total data 'HDD' 7,200 RPM SATA II storage.? Network performance is a big consideration.? My house is hard-wired for 100Mbps.? Software capabilities is another consideration.

------------------------
Jim W
Phoenix, AZ. USA

Losmandy G11G, NINA 2.0 Beta / ASTAP, ASI2600MC Pro, Sky-Watcher Scopes


 

I have a couple of QNAP nas devices -- with my Windows 2012r2 file server have roughly 36TB of accessible storage (I keep multiple copies as an alternative to backups). You will want gigabit connections -- AP raw images are huge. Started with 4TB drives, those are considered small now. While I used USB connected drives for years, I never found them truly reliable.


 



If it is just drives then higher end drives from the main manufacturers should be fine. I have had the Seagate Ironwolf and Barracudas in a Netgear ReadyNAS (not sure they are still made) for years without any real issues - caveat:- use Raid 5 or other as you will replace mechanical drives at some point in time.

+1 with Gregory's comment of Gigabit networking - along with a Gigabit and 10Gigabit interface on the NAS (a lot have both these days).

Good luck,
Ken


 
Edited

Jim,
After I got my DEC spike problem solved with my 811 (the key adjustment was to reset the tension on my SLW boxes, as per a Scott Losmandy video that Brian sent me - the adjustments eliminated the backlash and loosened the box so that it would float as designed - it was over tightened and frozen) I went back to AP adventures. Last month I wanted to try and put together a mosaic of the Cygnus Loop. I spent seven sessions gathering data for the four panels with my ASI2600 MC Pro. I am using NINA for acquisition and PI and a trial license with APP for mosaic processing. ?As I got into the processing, I found that my astro files (9 months of data and processing on about 35 DSO's) had maxed out my 2 TB SSD drive. I had a storage problem.

I began to read and watch videos and talk to a few nearby friends. It seemed I was in the market for a NAS. Most of the advice suggested that I get a unit that could handle five years of data collection as I look to the future. That made sense. They also suggested the 3-2-1 strategy, 3 copies of your data on 2 different media with 1 offsite. I thought that was probably a bit of overkill for me. A third piece of advice had to do with read/write speeds of hard drives and transfer speeds between my computer in the NAS unit. ?

I purchased a Synology DS218+ double bay unit, but swapped it out for a DS940+ 4 bay unit when I discovered the 218+ could not accept future expansion units. The setup was simple enough even for someone like me without any networking experience. I installed 2 10TB hard drives (leaving 2 bays available for future expansion), plugged it into my router, managed to set up a storage pool, a couple volumes, some folders, and transferred my Astro data over. I also ran an ethernet 6 cable from my router to my workstation. It all seems to work quite well, and I am pleased with it at this point.

I have had to make one important adjustment in my processing workflow. The Cygnus Loop mosaic project has been massive, with 1400 X 60s subs, gathered over seven sessions and stacked into four different panels. As you know, Jim, our ASI2600s produce 52 MB subs! Then there were the layers of folders to get from the subs, to integrated masters, through the assembly process.?I ended up with 28 masters for my panels that were 940 MB each, before I ever got to my a final master. ?When I tried to handle this workflow with my computer hardwired to my NAS unit, the data transfer was too slow. So what I ended up doing was transferring all the subs to my newly empty SSD drive and plugging it directly into my computer until my processing was completed. That resolved any transfer delay issues. After I am done, I will plug the SSD into the NAS UBS3 port and transfer all the data back to the NAS. Easy to do.?

I will be very interested to know how you resolve the data storage issue.

I am also very interested to know what data of people keep long term. I assume that I will be adding fresh data to my DSO folders in the future and will want to process everything again. I also assume that the software is developing so rapidly that reprocessing my data with the new software competencies will sooner or later be worth doing. So what should I keep? I would really prefer not to have to keep everything. Do I keep the raw fits files with the calibration data? I assume that I keep my masters. I noted that some people said they kept only their registered folders plus their masters. But if you wanted to preserve your ability to re-process, wouldn't you have to keep something earlier, like perhaps the calibration folders? I'd be glad for any advice.

Geoff Chapman


 

Hi Jim

boy i feel you on this one. I am also a longtime terrestrial photographer so I have a lot of storage needs but astro takes it to another level

I have a 45tb NAS from Synology, which is a base unit plus an expansion. However, I really find the disk speed to be a non-starter for active astro work. It just takes too much time to pull it across the network.?

I also have a 20tb DAS that I got for trying to improve the situation, it¡¯s a drobo attached via USB3 and even it is too slow

So my approach is to use a faster 4tb SSD for active sessions, and once it¡¯s done, i move the files i want to keep over to the NAS or DAS.

I really like both Drobo and Synology. I had a bad experience with losing a bunch of photos on the Drobo, so I now have battery backups on all those drives


Brian


On Thu, Jul 14, 2022 at 12:39 PM Jim Waters <jimwaters@...> wrote:
I am getting tired of USB3 SSD's and HDD's floating around that I use to store my PI Projects.? Its no problem for me to build a NAS but I don't have the time or parts.? I have no experience with off-the-shelf NAS Drives.? What are good manufacturers and models to start looking at?? I am currently checking out Western Digital and?Asustor.? What do other do to store images?

I would need RAID 1 (minimum) and about 6 to 8TBytes of total data 'HDD' 7,200 RPM SATA II storage.? Network performance is a big consideration.? My house is hard-wired for 100Mbps.? Software capabilities is another consideration.

------------------------
Jim W
Phoenix, AZ. USA

Losmandy G11G, NINA 2.0 Beta / ASTAP, ASI2600MC Pro, Sky-Watcher Scopes

--
Brian?



Brian Valente
astro portfolio?
portfolio
astrobin?


Jim Waters
 

Thanks all you your inputs.? I am not sure what I was thinking.? I really need 20 plus TBytes of storage.? I have ~1.5Tbytes of Canon 6D / 5DMkIV terrestrial images to store also.

For astro stuff I retain the Lights, Flats, Registered and PI Projects for each target.? That adds up fast.? Especially if I image from my backyard.? I usually have 150 plus Light subs.? I usually zip stuff up too.? RAID 1 is sufficient for my needs.? I may archive stuff that's more than 4 years old to USB3 HDD's.

The?Synology DS920+ is at the top of my list.

------------------------
Jim W
Phoenix, AZ. USA

Losmandy G11G, NINA 2.0 Beta / ASTAP, ASI2600MC Pro, Sky-Watcher Scopes


 

Good point, Brian.

You want UPS battery backup if for no other reason than to ensure an orderly shutdown and buffer flush. In my configuration there are two UPS units -- one feeds the NAS units, the network switch and firewall, the server is on the other. Enough battery capacity to run things for an hour. An orderly shutdown of the arrays and server starts after 30 minutes of power outage. The observatory PC has two SSDs -- the images from the night (or day) get flushed to the server while I am closing. Nothing is retained there -- it all moves to the network.

g


 

It is pretty scary just how much storage you need as an imager even with a modestly sized mono camera these days.?As a working photographer 10 years ago I really needed some long term storage so I built a Thecus 5-bay NAS with 20TB (5x4TB) of fast Seagate Barracuda drives. At the time the Thecus was very fast for a single 1Gbit Ethernet port NAS. Operating in RAID6 I only got about 12TB working space and it was slower when writing than RAID10 but the double drive redundancy was more attractive than the 2-3x faster write speed for a long term storage system. This was fine for the photography work I was doing with the 1Ds and 1Dx and my sports shooting was lower volume.?

Fast forward to 2022 and 12TB is a drop in the bucket for a serious astroimager. It seems like you will need to buy multiple NAS or really high capacity drives and at least 6-8 bays to last much time unless you toss out everything not 100% perfect.??


--

Chip Louie Chief Daydreamer Imagination Hardware?

? ?Astropheric Weather Forecast - South Pasadena, CA?


 

I've gotten by so far with a much simpler setup. I just have 2 5T usb3 drives network mapped. I have a nightly job rsync one to the other for backup. They're network mapped so I save to them from my raspberry pi during imaging. For processing I setup pixinsight to use a fast flash drive. The drives are half full after a couple years -- YRMV. By the time I need to upgrade I'm hoping that the cheap sweet spot size / price will be much bigger, and faster.?

Jamie?
--
jamie.amendolagine@...