¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io

3 series internal memory failure...AGAIN!


 

So after several processors having internal memory card failures, I have very little confidence in the reliability of 3 series processors. ?What is everyone else's experience with this? ?It seems that the failure will happen on systems that have been running fine for a long time and as soon as I upload to them, they slowly or all at once crap the bed. As far as I know Crestron hasn't solved the issue, and it's only a matter of time until every 3 series processor will have this problem. ?I tried loading programs to removable media, but found out that large programs take forever to start after a load when they are on RM. I had an MC3 for a large residential job (separate processor for lighting/ only running one program) that consistently took around 15min to stabilize after a program load. ?True Blue said this was considered normal. Anyone else have thoughts on this? ?


 

I haven't seen it happen yet on a CP3, but if it does it's going to be sweet when the 75 room identical room job I just worked on has all the processors fail at the exact time since they're running the exact same program, and then once they get repaired they will probably require $37,000 in licenses for the lutron integrations :)


Andrew Welker
 

There isn't really anything Crestron can do about this that they haven't already done. Flash memory of any kind has a limited lifespan because of the technology. If the program is throwing lots of errors and causing lots of writing to the plog files, that will shorten the lifespan of the processor. If the program uses config files and they are modified often, that will shorten the lifespan. I was watching the deep dive talk by Toine, and if I remember correctly, he said they planned for about 10 years on the SD card, but that things programmers do can shorten that lifespan.


On Mon, Jul 3, 2017, 11:43 PM cb950@... [Crestron] <Crestron@...> wrote:
?

I haven't seen it happen yet on a CP3, but if it does it's going to be sweet when the 75 room identical room job I just worked on has all the processors fail at the exact time since they're running the exact same program, and then once they get repaired they will probably require $37,000 in licenses for the lutron integrations :)


 

Then they need to make the SD cards field replaceable period. It's a complete joke that the entire processor has to be sent back to replace a $10 SD card. I've had a few MC3s sent back because of this. I was told that early versions of the SD cards were not setup correctly which is what causes the early failure. Crestron knows about this but has decided to just charge us for the mistake.?

I'm in Canada so it's tough to explain to the customer that it costs hundreds of dollars (labor and shipping charges) to replace an SD card.?


 

¿ªÔÆÌåÓý

There was also a firmware change to minimize the number of file writes when the log is ¡°chatty¡± ¨C if you have any processor running old firmware, updating the firmware will significantly extend the flash lifespan.

?

--

Lincoln King-Cliby, CTS, DMC-E-4K/T/D

Commercial Market Director

Sr. Systems Architect | Crestron Certified Master Programmer (Gold)

ControlWorks Consulting, LLC

Crestron Services Provider | Biamp Audia Certified | Extron Control Professional

?

From: Crestron@... [mailto:Crestron@...]
Sent: Tuesday, July 04, 2017 12:04 PM
To: Crestron@...
Subject: [Crestron] Re: 3 series internal memory failure...AGAIN!

?



Then they need to make the SD cards field replaceable period. It's a complete joke that the entire processor has to be sent back to replace a $10 SD card. I've had a few MC3s sent back because of this. I was told that early versions of the SD cards were not setup correctly which is what causes the early failure. Crestron knows about this but has decided to just charge us for the mistake.?

?

I'm in Canada so it's tough to explain to the customer that it costs hundreds of dollars (labor and shipping charges) to replace an SD card.?





 

"So after several processors having internal memory card failures, I have very little confidence in the reliability of 3 series processors. ?What is everyone else's experience with this?"

Been working with 3-series since they came out and have not had a single one fail with a memory card issue.? In fact, I'm not sure I'm aware of any installed processors needing to be replaced for any reason, period.

That includes the MC3 running my home AV system...

I do remember a story about a Crestron engineer that had to delay the start of their vacation to investigate mysterious issues at a large client's site.? Turns out the developer's code was writing to a file with some kind of unnecessarily high frequency and killing the cards in all the processors at the site...

? - Chip


 

Okay all -

Say it isn't so. I just wrote for 30+ rooms on a CP3N and PRO3 using the write to file Simpl+ module on this site.

What I am hearing is, that module will shorten the life of the rooms I just programmed because of a SD card failure?

I have always complained, even on Labs, why there was not a better way to write to NVRAM.

This is simply unacceptable.

AV Programmer.


Andrew Welker
 

Not necessarily. How often are you writing to the file? Is it once or twice a day? Not a big deal. Once or twice a minute? That's going to shorten the life, maybe from the designed to 10 years to 8. Again, this same kind of problem applies to any kind of flash memory. It has limited read-wrote cycles because of the technology. Its on the order of several million cycles, but it's not infinite.


On Fri, Jul 7, 2017, 11:43 PM av_programmer@... [Crestron] <Crestron@...> wrote:
?

Okay all -

Say it isn't so. I just wrote for 30+ rooms on a CP3N and PRO3 using the write to file Simpl+ module on this site.

What I am hearing is, that module will shorten the life of the rooms I just programmed because of a SD card failure?

I have always complained, even on Labs, why there was not a better way to write to NVRAM.

This is simply unacceptable.

AV Programmer.


 

I had a conversation with a TB tech and was told the problem with the early 3 series cards was they were not setup properly. He said instead of writing to the entire SD card they would write to the same block over and over again causing the cards to fail prematurely. This is a KNOWN issue with Crestron but instead of being proactive, they've decided to be reactive and just charge everybody for their mistake.?

The bigger issue was who was the idiot that didn't make the cards field replaceable???


Michael S. Moran
 

¿ªÔÆÌåÓý

I was the unfortunate recipient of that largesse. My processor was located in Maui...lovely place but some 19-20 hours away from my home base.?

Best regards,

Michael S. Moran

On Jul 8, 2017, at 11:17, kgossen@... [Crestron] <Crestron@...> wrote:

?

I had a conversation with a TB tech and was told the problem with the early 3 series cards was they were not setup properly. He said instead of writing to the entire SD card they would write to the same block over and over again causing the cards to fail prematurely. This is a KNOWN issue with Crestron but instead of being proactive, they've decided to be reactive and just charge everybody for their mistake.?


The bigger issue was who was the idiot that didn't make the cards field replaceable???


 

I'll offer some other insight on this.

Back when I worked for Crestron and was adding Fusion EM to the processors we found the NVRAM was failing. It was selected originally to be the location for the Fusion event XMLs and the temp storage for the energy logs.

Well after running for several weeks/months the events quite running and the energy logs stopped showing up in Fusion. They found the memory was "wearing out".?

The event XMLs contain 7 days of events in case connectivity is lost. It writes them as needed to keep them current. On the other hand the energy logs are done by default every 10 minutes the transferred to the Fusion server and deleted unless Fusion says bad XML. It is 1 file for each load every 10 minutes. If you lengthen the interval it just makes a bigger file and takes longer to upload.?

Using a high quality SD card seems to have resolved the issue of failures.?

If you should use the SD card for all frequent R/W things. NVRAM is fine to store preset lighting levels or other low usage data.?


 

Hey Guys:

This is all great information. As we all know, there are many types and qualities of memory.

SSD¡¯s and newer M.2 busses have really paved the way for great solid state storage devices that are low energy consumption and better life.

I¡¯m just wondering why Crestron went with the cheapest solution - SD cards - and not a more robust solution.

I¡¯m looking at installing a M.2 SSD into my laptop - a Samsung PRO 960 M.2 1TB is roughly $500 but the speed is incredible and promises longer R/W life.

Crestron would have been better served to use a small SSD hard drive instead of cheap SD cards.

By the way - I am on a campus where we had TSW-1050 & 1052, TSS-752 failing everywhere. I¡¯m still requesting SD card replacements. I know, update the firmware and all is well.

Enough of my rant - this was a great thread! If anyone else has more information to contribute - please do. When things fail, my experience is they always call the programmer.

AV Programmer -


Emperor Bob
 

¿ªÔÆÌåÓý

Honestly, ?I don't think there's room in TSW models to use a SSD. ?It's no different than Cisco using SD cards or using NVRAM chips soldered on boards. ?All chips go bad eventually, ?it's just a matter of time. ?Now had they mad them externally accessible, it'd be much easier.


 

If my memory from Master's class serves, Toine talked briefly about this, and mostly warned about programs that write info to logs, as some people are writing to log files every few seconds or faster and this can really reduce the life of the memory chip.

Every few minutes is better, and the longer you can go between writes gets better and better.? He was warning about sub minute increments that really effects the life of the memory chips.

As others have mentioned, the memory is only good for so many writes (SD cards are not the only memory with this limitation, SSD drives have limits as well).

Doing some basic calculations, using 1M write cycle estimates, and you can see the drastic difference of what these write cycles can mean to life expectancy of the memory chips (ignoring other write cycles that not under user control, such as errlog and program loads):

- Write to file in 30-second intervals:? life expectancy LESS than 1 year!

- Write to memory once per minute, you will hit the 1-Million marker in LESS than 2 years of operation.

- Write to file every 5 minutes:? You get about 10 years before you hit 1 Million cycles.

- Now if you write to the log every 10 minutes:? you won't hit the limit until almost 20 years.

- Once per hour?? You won't hit the limit for over 114 years!


Jason
www.mpsav.com


 

This is a great post and puts read/write into perspective.

Is there any official document like this or was it Toine @ Masters during a session?

I have been doing AV over two decades and sometimes my filters are nonexistent when it comes to things like this - but why isn¡¯t there an official document that speaks to this?

I know many TB technicians, and this forum is much more informing than any TB call I have been on. Frankly, I find TB usuful only for RMA items and that is sad to say.

Now, Extron has incredible customer support. But they are from two different mindsets - Andrew is from L.A. and then you have the hit men Jersey Boy style. Andrew Edwards is a great leader and understands customers - Crestron is something different.

Crestron - if you are reading this, take some lessons from Extron concerning customer service and support. And please stop making me feel like my Crestron issues are my fault.

By the way - we just lost another A+ Rep in our area. He was a stellar guy but Crestron kept beating him up over stupid decisions that Jersey was making - like the Fusion Cloud debacle! I understand people who start working for Extron stay awhile.

AV Programmer -


 

Don¡¯t think anyone at TB ever went postal though . . .

From: <Crestron@...> on behalf of "av_programmer@... [Crestron]" <Crestron@...>
Reply-To: <Crestron@...>
Date: Sunday, July 9, 2017 at 1:34 PM
To: <Crestron@...>
Subject: [Crestron] Re: 3 series internal memory failure...AGAIN!

?

This is a great post and puts read/write into perspective.

Is there any official document like this or was it Toine @ Masters during a session?

I have been doing AV over two decades and sometimes my filters are nonexistent when it comes to things like this - but why isn¡¯t there an official document that speaks to this?

I know many TB technicians, and this forum is much more informing than any TB call I have been on. Frankly, I find TB usuful only for RMA items and that is sad to say.

Now, Extron has incredible customer support. But they are from two different mindsets - Andrew is from L.A. and then you have the hit men Jersey Boy style. Andrew Edwards is a great leader and understands customers - Crestron is something different.

Crestron - if you are reading this, take some lessons from Extron concerning customer service and support. And please stop making me feel like my Crestron issues are my fault.

By the way - we just lost another A+ Rep in our area. He was a stellar guy but Crestron kept beating him up over stupid decisions that Jersey was making - like the Fusion Cloud debacle! I understand people who start working for Extron stay awhile.

AV Programmer -


 

We had TSW 752s failing all over campus until we did the latest firmware update. ?Since then they have been pretty reliable.


Joe Vossen
 

In what manner were your 752s failing? I have one that reboots itself about every 12+ hours and nothing shows in the logs. The logs in the switch it is connected to shows link going down/up twice in rapid succession; the 752 acts as if power was pulled and restored.

Thanks

On Aug 30, 2017, at 2:43 PM, chuck3564@... [Crestron] <Crestron@...> wrote:

We had TSW 752s failing all over campus until we did the latest firmware update. Since then they have been pretty reliable.


 

Ours would have a black screen and they would usually come back if you did an initialize and restore. ?After the firmware update we have had one or two, after sitting for a month, do the same thing. These came back with just a power down and up.


 

So, if I understand all of you correctly, the NVRAM DISK feature is store in flash instead of Non Volatile RAM -- e.g. static RAM with a battery or super cap for storage backup?? I always thought the latter was true or was that just the case with Series 2 processors.?

Because static RAM storage is usually a lot smaller than flash, I thought that was the reason why the NVRAM DISK feature was limited to 64 or 128KB.?

If Crestron is using flash for the NVRAM DISK feature, they should rename it NV FLASH DISK.?

Maybe it would be better to use an external SD card or USB memory stick.? At least you can replace that.

Thanks,
Steve