Someone that knew Bill Gates when he was a boy in on the podcast this week, although we only talk about Bill Gates for a moment. He has 30+ years in backup experience and tells us what it’s been like to adapt to all of the backup changes over the years. His first backup was to punch cards and punch tape, and he was in the same Boy Scout troop as Bill Gates. Fans of the podcast know his name already, as it comes up randomly on the show as a friend of Curtis. But this is the first time Stuart Liddle has graced us with his presence.
Like Mr. Backup, his career starts with a data loss story that actually involved people having to re-enter data. We discuss a lot of configuring and running backups, including deciding on retention periods, treating all backups the same (or not), virtual tape libraries and other dedupe systems, and how important a change management data database (CMDB) is. We also talk about the danger of becoming entrenched in a specialty like backup, knowing only one specialty or product. We talk about how it’s not good for you or your company.
Finally we talk about the different way people are using the cloud today for IT and backup, and how that affects cost. Curtis and Prasanna use a great analogy that helps it make sense.
This week’s episode is fully of useful information.
Video
Transcript
[00:00:00] W. Curtis Preston: The only advantage of the Uber in that scenario is you can get drunk and not worry about it, but I don’t think the Lambda function, you know, is gonna help with that.
[00:00:30] W. Curtis Preston: Hi, and welcome to Backup Central’s Restore it All podcast, I’m your host, W. Curtis Preston, AKA Mr. Backup and have with me, my garage lighting consultant, Prasanna Malaiyandi. How’s it going, Prasanna?.
[00:00:43] Prasanna Malaiyandi: I’m good, Curtis. Although I think shortly you will become my garage lighting consultant.
[00:00:48] W. Curtis Preston: You think, so you think it’s gonna be like a, like a, what do you call it? A, the tables will turn. You think
[00:00:54] Prasanna Malaiyandi: Or it’s like when the, what is it when the student becomes a master
[00:01:02] W. Curtis Preston: Yeah, this is, this is not one of those areas though. Where you, where you, you know, pulled out one of your, your extensive YouTube experiences, as I
recall,
[00:01:14] Prasanna Malaiyandi: so I think it’s fair to say that you did this project and then you talked to me after it was all done
had you reached out before I would’ve? Yeah, I’ve done quite a bit of research on garage lighting, so
[00:01:26] W. Curtis Preston: of course you have, of course you have, how many YouTube videos have you watched on garage lighting?
[00:01:36] Prasanna Malaiyandi: At least 30
[00:01:40] W. Curtis Preston: This is what I’m talking about. We’re back, we’re back to you being a random expert in stuff that I have no idea why you’re an expert in, but yeah, I did. I did this when a little willy-nilly I just sort of, I, you know, I’ve been needing that light in there. This is in my woodworking room that as you, that you know, that I have built and now, and I, I, I figured that it would be a good idea if there was good lighting in a place with all
[00:02:09] Prasanna Malaiyandi: Sharp tools.
Yeah. I was actually gonna think if you need like, uh, task lighting as well, or if there’s enough, because depending on like, you don’t want the light behind you when you’re on a table, saw trying to cut a piece of wood, you know, you’ll get the
[00:02:23] W. Curtis Preston: I usually close my eyes when I use a table saw. Should I not do that?
[00:02:27] Prasanna Malaiyandi: And this is like, one of your fingers is
[00:02:32] W. Curtis Preston: That one, there you go. That finger I’m not flipping ’em off viewers that are viewing both these fingers have been injured in power tools. My wife gets very angry at me when I display my, my injuries. Um, and yeah, so this finger, uh, it actually had an evulsion, which is a, a laceration that removes flesh. Um, that was not good.
So, yeah, that was many years ago, you know?
And
it’s been an awfully long time.
Yeah.
Learned a lot. Uh, let’s see. The last power tool injury was only. A, um, a nail gun, a shooting, a nail into my head.
[00:03:19] Prasanna Malaiyandi: and that’s been a couple years. Right.
[00:03:21] W. Curtis Preston: yeah, it’s been a couple of years. Yeah. And, and the, the hilarious part was I did it, like, after telling my granddaughter don’t touch this tool because it could do really scary
[00:03:30] Prasanna Malaiyandi: yeah.
[00:03:31] W. Curtis Preston: And then I send it, I try to send a nail through my, I don’t think it made all the way through, but it definitely cuz I was holding a piece of wood and shooting a nail into it.
And the, the nail went, it took a corner,
the nail, it bent a corner and it went into my
finger.
[00:03:54] Prasanna Malaiyandi: Yeah. Always be careful around power tools and make sure
you wear your safety goggles.
[00:04:03] W. Curtis Preston: All the things, um, you know, always back up your data always wear your safety glasses and um,
[00:04:11] Prasanna Malaiyandi: sure you have plenty of lighting.
[00:04:13] W. Curtis Preston: we ha we make sure you have plenty of lighting. We have a, another cool guest today. This is one of those people. Another person who, uh, he has a unique vantage point in that he has known me for a really long time.
He first got to know me as a customer, meaning he was a customer of mine. I was a consultant, um, at a company that he used to work. He, now I am super jelly. He is now retired and, um, is like, you know, hopping on cruises left and right. So I really, I dislike this guy, but he’s but he’s a really good guy. And, uh, you know, he’s been around like me a long time.
So, you know, he’s got a few backup stories to tell, welcome to the podcast, Stuart Liddle.
[00:05:12] Stuart Liddle: Hi, how are you doing? Thanks. Good to be here.
[00:05:16] Prasanna Malaiyandi: I have
a
[00:05:16] W. Curtis Preston: and by the way that, yeah, go ahead. Go ahead. Go.
[00:05:19] Prasanna Malaiyandi: I have a question for you Stuart, when you retire from the backup industry, right. Is it kind of like a spy where it never gets turned off? Like those skills that you’re always thinking about when you’re on like a cruise it’s like, I wonder if they back up this data, what would happen?
[00:05:35] Stuart Liddle: Yeah. I, I, I have those thoughts from time to time, but you know, that it’s kinda like, uh, a headache. I wish it would go away real quick. you know, but, but yeah, I mean, I’ve been, I’ve been doing it since like the early seventies and I started out as a programmer in the Air Force because I was gonna get drafted.
and I signed up for the Air Force, which may or may not have been a mistake, but
[00:06:08] W. Curtis Preston: They still had the draft at that point.
[00:06:11] Stuart Liddle: when I was
when I went
in. Well, yeah, I mean, I got, I got my draft notice the day after I, uh, signed up and that was in 73, 72 or 72 72. Yeah.
[00:06:29] W. Curtis Preston: So for the record, um, I was in like second grade, I throw that out there.
[00:06:36] Stuart Liddle: and
so, so I
[00:06:38] W. Curtis Preston: that I get to feel young Stuart, so,
[00:06:41] Stuart Liddle: there. Okay. There you go. Um, so make me feel old. Yeah, that’s great. Um, so, so yeah, I was doing cobalt programming on punch cards when I was in the Air Force. So, and our, and you know, and our backups of those programs were the punch cards that we used. And God helped you if you dropped them.
[00:07:03] Prasanna Malaiyandi: would they actually make two copies of punch cards? Like if they,
[00:07:07] Stuart Liddle: You know, that never happened, but well, they, they didn’t do that. I mean, I, it, you know, I wasn’t thinking backups at the time. So if you, if you had a program that you’d written, um, you had the punch card deck, you could make changes to it, and then you could load it into the, into the computer and then you could put it on tape if you wanted to.
So that could be, you know, and I’m talking real to real, the stuff they showed .
[00:07:34] W. Curtis Preston: Did you ever, Did you ever, use punch tape?
[00:07:37] Stuart Liddle: I did. Yeah. I used that as well. Um, and that was actually cooler, you know, for doing, doing quick stuff. Oh, okay. Quick story. Quick story. When I was in high school, I, I went to a high school with, in Seattle with Bill Gates’ older sister, and she and I were in the same calculus class.
And Bill Gates and I were in the same boy scout troop,
[00:08:06] Prasanna Malaiyandi: Oh
[00:08:06] Stuart Liddle: and yeah, really? Yes. Yes. And, and Bill Gates went to a school called Lakeside, which was in the Seattle area, but he had a, um, there was a place in Seattle called Computer Center Corporation, which had some DEC, 10 gear and stuff like that. And my high school had a computer programming class and our teacher took us to that place.
And we did some, um, I think it was, um, Fortran programming or maybe basic. And we were using punch tape at the time for that. But, but when we went in there, apparently that was one of bill gates Hangouts, and he was not well thought of by the people that managed that data center, because he would typically come in and do things to make their lives miserable, like crash the computers and stuff like that.
So just, there’s a bit of trivia for you. Yeah. Yeah. Typical high schooler, but look where he is now, you know, a divorced billionaire
that’s about, as much in the way of bill gates stories I can tell.
[00:09:19] W. Curtis Preston: That’s interesting.
[00:09:21] Stuart Liddle: We talked earlier about, um, something that happened very early on in my after military career,
um, which was, I went to work for a large bank in Seattle that doesn’t exist anymore and not because of this incident, but, um,
[00:09:41] W. Curtis Preston: That’s good to hear.
[00:09:42] Stuart Liddle: They, they, uh, they had, they had a situation that one of the guys came to me one time and I hadn’t been on that job for very long, maybe a month or two.
And, um, this guy came up to me and he says, Hey, we need some help. Um, we just had the last tape copy of this information get overwritten. they only had three copies, you know, the. Grandfather father and son, whatever. I don’t know what, whatever they wanted to call it, but, but in any case, they needed to have this data brought back and they asked for me to help with it along with a couple of other people.
And basically the only information they had on that data was all on printouts. You know? So here’s the latest printout that we have and here are the columns of data and whatnot. Can you help us out? And we had to figure out a way, okay, we need to have data entry people type in stuff from the printouts and then put it together in the right order and stuff.
Yeah. So that was my first, I guess, data recovery project that I ever worked on.
[00:11:02] Prasanna Malaiyandi: And how long that
[00:11:03] Stuart Liddle: never thought
of it. I, it only, I think it only took us. Um, well I think the data entry took a while. I don’t recall exactly about that, but I know that the people that I was working on this with, we each had a piece of it to write, to pull the data from whatever was gotten in through data entry, into, um, a, the, the format that it needed to be in.
I, and I, and I think that part may have only taken a couple or three days, but the actual data entry part could have taken quite a bit longer, but we were, we were able to get it done. So
[00:11:42] W. Curtis Preston: So they, so they had multiple tapes of it, but somehow all three had been
[00:11:47] Stuart Liddle: overwritten
Yeah. Yep. And it was, and it was based on a certain period of time, you know? So like after seven days the grandfather got over it and then after 14 days, the father got over that kind of thing.
[00:12:04] W. Curtis Preston: And then it’s funny, that is not a data restoration story. That is a data reentry story. that’s like the worst case scenario, right.
[00:12:15] Stuart Liddle: right.
[00:12:18] W. Curtis Preston: Yeah, well, we don’t have printouts anymore. Nobody’s gonna be a printout. Um, unless you’re like a, a CPA, they like to print out paper and have it hanging around. Um, but yeah, it, it reminds me of my, my first data, you know, restoration nightmare, because.
[00:12:41] Prasanna Malaiyandi: Oh,
[00:12:42] W. Curtis Preston: also had, when we had this outage, it was, it was, it was a purchasing database. And when we had the outage, I went to go look for the tapes. And I was told to look at the logs. It, they, that these were unique systems and this was dump. And I was told to look at the logs and I was told to look for these certain errors.
And if these errors were there, then go to the previous backup. Well, I kept going and I kept going and I kept going and the errors were always there. And, uh, eventually I found a backup that was, you know, that was valid and it was just over six weeks old. And, uh, our retention period was six weeks. Right.
And, you know, I remember just wanting to crawl under, and, and that, you know, speaking of speaking of dudes going on Alaskan vacations, uh, this morning, I’m literally looking at on Facebook. Another friend of mine, that is the guy that saved my bacon on that day. His name’s Joe Fitzpatrick, another guy whose name comes up once in a while in the podcast, he doesn’t even remember this event.
Right. Um, I just remember that he, he pulled up the Unix, you know, command prompt, which at that time for me was still like magic. I was only in maybe a couple of months and he did some stuff with F S C K or something, and somehow got the drive to a point where we could get something off of it. And we only lost.
Two days worth of purchase orders in this database. And to me, that became, you know, the, the, the defining moment in my career. Like, I’m like, that’s not gonna happen to me. And to Joe, it was just a Tuesday. So to him, it doesn’t, he doesn’t even remember this story. It’s like this single handedly, like the, the single most defining moment of my career and Joe doesn’t even remember it.
And now he’s off, he’s off to Alaska sending me pictures. Like you’ve been sending me from your Alaska cruises
[00:14:48] Prasanna Malaiyandi: Yeah.
[00:14:49] W. Curtis Preston: so
[00:14:49] Stuart Liddle: So
[00:14:52] W. Curtis Preston: do you remember the first time we worked together?
[00:14:56] Stuart Liddle: yeah. Yeah, it was, it was at Amgen.
Um,
[00:15:00] W. Curtis Preston: And was Thousand Oaks
[00:15:01] Stuart Liddle: to help
[00:15:01] W. Curtis Preston: or was in Seattle?
[00:15:03] Stuart Liddle: It was in thousand Oaks.
[00:15:04] W. Curtis Preston: Okay. All right.
[00:15:06] Stuart Liddle: Well, okay. Wait a minute. Wait a minute. You’re right. There was a class that you did in Seattle.
To the people that were there just after the merger.
[00:15:19] W. Curtis Preston: Right. They had, yeah, it would’ve
[00:15:21] Stuart Liddle: yeah. And so you did a class on NetBackup and because they were using NetBackup at, um, Amgen and you did a class there on NetBackup.
I remember. I, I think that’s what happened.
[00:15:35] W. Curtis Preston: yeah. The, the, the funny thing is, you know, like I was not a NetBackup instructor, but it was like, this is how I use NetBackup. That’s what I remember if I was gonna
do.
[00:15:46] Stuart Liddle: That was in 2002, I believe.
[00:15:50] W. Curtis Preston: Oh, it was a few years ago. How do you remember that year? I’m so bad at years.
[00:15:56] Stuart Liddle: were, there were certain things that happened in certain years. I mean, that’s, that’s, that’s one of ’em, you know, it’s like, that’s when I, as part of that merger, they took a group of people at, at the Seattle office and they said, You can either go to work for us in thousand Oaks or you can get laid off in October.
[00:16:21] W. Curtis Preston: You’re like, I think I’m gonna move to Oxnard Camarillo.
[00:16:24] Stuart Liddle: yeah, well actually it was thou um, uh, Camarillo, but that’s okay. Yeah,
yeah, yeah. Close enough.
But, um, yeah,
[00:16:35] W. Curtis Preston: um,
[00:16:37] Prasanna Malaiyandi: what was Curtis like back then? I only know Curtis now, what was he like back then?
[00:16:42] Stuart Liddle: he was, he was just as funny and, and interesting as he is now. I’ll I’ll, I’ll just go
with
[00:16:49] W. Curtis Preston: I was I as devastatingly handsome as, as I am now,
[00:16:53] Stuart Liddle: Well, you know, I’m, I’m not the person who would be able to give a, a good, uh, assessment
[00:17:00] W. Curtis Preston: way, speaking of being devastatingly handsome, we are the bearded trio today. The bearded and headphones trio, two of us with gray beards one, not so much, although there’s some
[00:17:14] Prasanna Malaiyandi: great.
[00:17:16] W. Curtis Preston: There’s some gray in there. Um,
Thinking about the I’m pretty sure it was the Seattle days. There was a boss and I think you and I have talked about this recently, but there was a boss that you had, and I’m pretty sure he was there and, or in Thousand Oaks. Darn if I can remember, but the name, the name, Scott rings a bell and he was the one that he wanted backup retention of two weeks and no more.
Do you remember this?
[00:17:48] Stuart Liddle: I, I don’t remember that. I know the guy you’re talking about and actually he, he was one of the people who was involved in the transition from the company that I used to work for that got a, um, merged into Amgen. And he later became a director in Seattle and then went back to Thousand Oaks, um, you know, at a higher level.
[00:18:11] W. Curtis Preston: But you don’t, you don’t remember this thing about the two weeks?
[00:18:14] Stuart Liddle: I don’t remember the two weeks. I don’t remember the
[00:18:17] W. Curtis Preston: cuz I remember it very distinctly cuz it’s the only time in my entire career when I had someone say something that to me sounded so ill advised
[00:18:29] Stuart Liddle: Yeah. Well,
okay. Does, does that sound as ill advised as treating all backups equally?
[00:18:41] W. Curtis Preston: um, I I’m gonna say more ill advised, but I, I know what you’re saying about. Yeah. So is that, was that something else that you ran into? Hmm.
[00:18:49] Stuart Liddle: Oh, I ran into that when I was consulting, I found a, you know, there was a big, um, oil and gas company in Texas that , that said dev test and prod. It doesn’t matter. It all gets treated the same. And I’m like, you guys are insane. This is, this is
crazy
talk.
[00:19:07] Prasanna Malaiyandi: if I just
have the,
budgets.
[00:19:10] Stuart Liddle: yeah,
[00:19:11] W. Curtis Preston: I mean, as long, yeah, if you don’t care about money, number one, and number two, if you basically, as long as you’re not treating dev test and prod like dev, right. As long if you’re treating dev test and prod like prod,
[00:19:29] Stuart Liddle: mm-hmm
[00:19:30] W. Curtis Preston: you know, and you have a solid plan and everything for everything, but I would still that even if you, even if you’ve got a system that’s capable of backing up and restoring all of that, you at, you still need priorities
[00:19:46] Stuart Liddle: Yes.
[00:19:46] W. Curtis Preston: in, in a disaster recovery situation, you’re not gonna recover dev and test before you recover
prod.
[00:19:54] Stuart Liddle: Right.
[00:19:55] W. Curtis Preston: even prod, you’re, you’re not gonna recover all those systems at the same time.
You should have segments, right.
Um, Yeah,
[00:20:04] Stuart Liddle: Like, so, I mean, you know, another part of that is they didn’t have a, uh, a CMDB,
[00:20:13] Prasanna Malaiyandi: So what’s a CMDB for our listeners. Yeah.
[00:20:15] Stuart Liddle: Configuration management database. In other words, you got all your servers and all your applications in this database, and you say what they are, whether they’re dev pros, prod, or test, and then you. You prioritize them like, like Curtis was saying, and you can, you can find them because you know, their location, you know, their name, you know what they do, all of that stuff.
Um, you got serial numbers on all of the hardware, asset tags on all the hardware. They didn’t, they didn’t really have that. They had, they had it, but they didn’t, it wasn’t fully populated with all the stuff they really needed. So
[00:20:51] W. Curtis Preston: so, Hmm. I have another distinct memory from someone in
[00:20:57] Stuart Liddle: Oaks. Oh,
[00:20:58] W. Curtis Preston: No from someone in Seattle, again, I could be wrong, but my memory is that it was someone in Seattle talking about migration. You were migrating off of Netware. Um, and there was a guy that was a Netware lover. And I remember
[00:21:17] Stuart Liddle: about too.
[00:21:18] W. Curtis Preston: I remember him saying that it was job security, that he, he knew, he knew that it was way harder to use than everything else and that nobody else knew it, but him, but he didn’t see a problem with that.
[00:21:33] Stuart Liddle: I know the guy you’re talking about and he, he actually, uh, bought a place in Idaho when I was, you know, before I moved there. And, uh, yeah, we’ve, we’ve kept in touch, but yeah, I, I know his name. Yeah. Old
[00:21:47] W. Curtis Preston: but yeah, but we see that, we see that a lot in, in. Technology. We see it in backup systems where someone a, a new and interesting and possibly much better backup system comes along. But then you got the guy that knows NetBackup. And so he doesn’t wanna let go. He’s got all this, you know, knowledge of having run the, the backup system for so long.
Um, yeah. Uh, wait, that guy was you I’m talking about you
[00:22:20] Prasanna Malaiyandi: but it doesn’t even have to be just backup. So even
[00:22:24] W. Curtis Preston: no, no.
[00:22:25] Prasanna Malaiyandi: moving to like VMware, right. Virtualization and your server admins are like, what am I gonna do now?
[00:22:30] Stuart Liddle: Yeah.
[00:22:36] Prasanna Malaiyandi: Yep.
[00:22:42] Stuart Liddle: oh, oh,
oh, that’s right. So is, is it going to,
[00:22:48] W. Curtis Preston: It’s happening.
[00:22:49] Stuart Liddle: it gonna be like Symantec and, and, uh, Veritas
[00:22:53] W. Curtis Preston: I sure as hell, hope not. Uh, cuz I re I remember, I remember going to Veritas Vision. Back when Veritas had a vision, they went to Veritas Vision, it was a year after the acquisition. And I remember saying it appears that you’ve spent an entire year painting everything yellow and black. Right. That’s literally, that’s literally all that they had done for the entire year.
[00:23:21] Stuart Liddle: Oh, probably
[00:23:22] W. Curtis Preston: And, and I, I, I don’t think I was being unkind. It is just that, that, that did not go well, like, and you know, and they parted ways and now there’s Veritas and now there’s Symantec.
I’ll throw out our disclaimer Prasanna. And I work for different companies. Prasanna work for zoom. I work for Druva, and this is not a podcast of either company. The things that you hear on here are our opinions, nothing more, and, uh, be sure to rate us ratethispodcast.com/restore.
And also, you know, if you like to talk about backup, restore, archive, security protection, privacy, cetera, we’d love to have you on just, uh, contact me at wcurtispreston@gmail or @wcpreston on Twitter. You can DM me. I accept all DMS and, uh, we can chat.
Let’s talk about some other interesting things that you remember over the years in terms of adapting to new, you know, new ways of doing backups or, you know, cuz you’ve, cuz you’ve seen and in 20 years you’ve seen the move from tape to, well from, from, from punch card to tape, from disk from disk to cloud, you know, what was that like?
[00:24:34] Stuart Liddle: Um, you know, it’s interesting when I was, let’s just take Amgen as an example. When I first got there, everything was going to DLT and we were using, um, a certain offsite storage vendor to, you know, pick up the things and take ’em off to some other site. Um, and then it went to, um, uh, so that was physical tape.
And then we went to virtual tape.
[00:25:04] W. Curtis Preston: Oh, yeah. Okay. Hang on. So DLT for the, for those of you that, that haven’t been around, that was digital linear tape. That was the predecessor to LTO. So it was a, it was a linear tape. The way LTO is as opposed to heli scan tape, which is a completely different thing. Did you ever use any helical stuff like D DDS or eight millimeter, uh, or AIT?
[00:25:30] Stuart Liddle: we, we did have some, and, and at the same time we transitioned off of DLT to LTO. We transitioned off of those as well.
[00:25:40] W. Curtis Preston: Gotcha.
[00:25:41] Stuart Liddle: We did have AIT. Yeah. So we transitioned to that. And then we also had gotten into virtual tape libraries and
[00:25:51] W. Curtis Preston: which
[00:25:51] Stuart Liddle: big vendors.
[00:25:52] W. Curtis Preston: if you’re new to this world, somebody’s listening to this going. I’m sorry,
what what is a, what is a virtual tape library? Stuart Stuart.
[00:26:05] Stuart Liddle: it’s a disc based system that emulates writing to tape and it
[00:26:10] W. Curtis Preston: And And why in hell would somebody do that?
[00:26:14] Stuart Liddle: supposedly it was faster and easier to work with, but
[00:26:21] W. Curtis Preston: but but why would a disc that I get, why you’d use disc? Why would a disc pretend to be tape?
[00:26:30] Stuart Liddle: Because the backup software primarily dealt with tape and didn’t know about writing to disk
[00:26:35] W. Curtis Preston: Yeah, it was a, it was a bandaid situation. Right. And I remember many years ago, several years ago, I was sitting, sitting, chatting with a gentleman by the name of Mark Staimer. And I’ll have to, I’ll have, now I’ll have to tweet to him about this episode, but Mark Staimer and I made a bet one year, and this was when I was a big fan of VTLs at the time.
And he’s like in five years, Nobody’s gonna be buying VTLs like it was something he’s like, there’s gonna be fewer VTLs sold than, than the alternative. And I was like, I bet you, you know, we, I, as I recall, I, I, I owe him dinner. He is the thing was that basically it was a time when backup products really didn’t know how to handle writing to disk very well. And so we needed to, and I, and I can think of one big backup. I’m gonna call it protocol that really didn’t know how to write to tape. And it’s one, I’m sorry that didn’t know how to write to disk. And it’s one that you would’ve, you would’ve been exposed to there at Amgen, cuz I remember helping with it. Do you remember it?
[00:27:50] Stuart Liddle: Tar
[00:27:52] W. Curtis Preston: No,
[00:27:54] Stuart Liddle: no, that’s right. You can write the
[00:27:55] W. Curtis Preston: I’ll give you a hint. Uh, the thing that used it rhymes with uh, Schmett app.
[00:28:06] Stuart Liddle: okay. Yeah.
[00:28:08] W. Curtis Preston: Do you remember? Do you remember? the way you backed up? What was that?
[00:28:13] Stuart Liddle: Oh, geez.
Now. Yeah. Okay. Yeah, there you go. There you go. That’s right. NDMP didn’t know how to do anything else, but
that yeah.
[00:28:22] W. Curtis Preston: right.
[00:28:23] Prasanna Malaiyandi: the only way to get data off your net app. Yep.
[00:28:26] Stuart Liddle: Yep.
yeah.
[00:28:27] W. Curtis Preston: And so, and so disc pretended to be tape and I, I think, I think they still sell that stuff, but I don’t, I mean, I know that quantum does, I know that, um, and does data domain still do,
[00:28:43] Prasanna Malaiyandi: Data Domain still supports VTL. Yep.
[00:28:47] W. Curtis Preston: for all three of its customers, that uses it.
[00:28:50] Prasanna Malaiyandi: no, I, so when I, so I used to be at EMC at the time, right after the data domain acquisition, I joined EMC. And I remember hearing stories though that the big selling point for data domain initially, like you’re saying, it’s you go into those accounts with tape customers and be like, Hey, here’s a very simple, de-duplicate applying deduplication appliance. You just swap out your tape drives or tape library with this. And you’re all good to go and look at these amazing cost savings that you get all the
deduplication.
[00:29:23] W. Curtis Preston: So that, that was the next phase, right. Was,
[00:29:26] Stuart Liddle: Yeah.
[00:29:27] W. Curtis Preston: was
[00:29:27] Stuart Liddle: Going to yeah. And,
[00:29:30] W. Curtis Preston: And
you, you used a lot of that stuff back in the
[00:29:32] Stuart Liddle: oh yeah, we used a lot of that stuff. We went from, we went to data domain and uh, then we ended up with strictly, uh, NetBackup appliances, so
[00:29:45] W. Curtis Preston: with, with NetBackup, dedupe integrated into it.
[00:29:49] Stuart Liddle: yeah. Yeah. And then replicating to a remote site using that.
[00:29:55] W. Curtis Preston: Right, right
[00:29:56] Stuart Liddle: So it.
it it basically eliminated the need for sending stuff off site to, you know, with tape
[00:30:04] W. Curtis Preston: So you actually lived what, what, I’m gonna call backup Nirvana. Right? Because we talked about this a lot, but I, but I very rarely met people that actually did it. And that was that they went tapeless with a de-duplicated replicated system. You had the onsite back and the offsite back and nobody touched a tape and no, man, there was no man in a van.
[00:30:29] Stuart Liddle: But that was only good at certain sites that would’ve allowed it due to bandwidth issues or, um, you know, in certain, uh, European states that wouldn’t necessarily allow it.
[00:30:46] W. Curtis Preston: What, and what was the challenge there?
[00:30:49] Stuart Liddle: Um, I, I believe there was like, um, in Europe and in the EU, they have certain data, uh, privacy laws that don’t allow you to move things over across borders or something
[00:31:03] W. Curtis Preston: Oh, okay. Well, I, I would just assume that you would do it within the border, but yeah, that, that’s definitely a
[00:31:09] Stuart Liddle: Well, If you like Switzerland, not part of the EU and has their own set of laws on that stuff. I think they’re, they’re one of the ones that we couldn’t really do it to, unless we had two sites in Switzerland, which I don’t think we did. We only had one. So, but
inside
[00:31:23] W. Curtis Preston: might be joining NATO though.
[00:31:26] Stuart Liddle: there you go.
[00:31:30] W. Curtis Preston: I think there’s a good shot there. Um, okay. So yeah. So if the laws allowed it, uh, and things like that, then you would do that.
[00:31:40] Stuart Liddle: Mm-hmm
[00:31:42] W. Curtis Preston: um, and what, how much better was that from comparing it to the tape world? To what you, what you had before, you know, so you had the tape world, you had the tape plus data domain world.
And then you went with this appliance world. How much better or, or was it better? Was it
more
[00:32:01] Stuart Liddle: Yeah. Huge, huge amount better. Um, the, the big problem with doing restores of, of files of course, is bandwidth across the wan. How long is it gonna take to restore a big file? That kind of thing. But I think it’s still worked out to be better than say, and, and we always did have the option. Here’s the option of at the remote site, if that’s the only copy of the data, copying it to tape and shipping the tape to the, to the site where you really needed to restore it, you know, that that cuz never underestimate the, uh, the bandwidth of a 747 full of DLTs or LTOs.
[00:32:40] W. Curtis Preston: That’s right. That’s right. Never underestimate the B bandwidth of a truck is my usual thing, but yeah. But yeah, plane full of
LTOs is pretty huge.
[00:32:52] Stuart Liddle: Yeah. Yeah.
[00:32:54] Prasanna Malaiyandi: was there something you found though that was lacking with this new solution versus what you had before? Cause I think everyone understands the pros of going, what sort of a backup appliance based approach versus older tape style. But was there something that you found was lacking with this approach or that you wish it could do that?
[00:33:15] Stuart Liddle: um, I, I think the only thing I wish I could do is to, to, to be able to be more of, um, quick restore across the WAN is, is probably the biggest thing. But for the most part, yeah, it’s, it was pretty nice having that because you didn’t have to deal with somebody constantly saying, well, I, I forgot I couldn’t get into the office because of the snowstorm.
So I couldn’t get the tapes offsite, and then you have a disaster, right. Just
don’t have to worry about that.
[00:33:48] W. Curtis Preston: you were able to have different retention on your primary and then your secondary, cuz you, you mentioned that there would be backups. It would only be at that secondary cuz you could solve the problem by having the same size systems on both sides. Right? Cause
then if you’re doing a restore from the, the place where you’re backing up, you would restore from the,
[00:34:09] Stuart Liddle: Yeah. You could
restore from the
[00:34:10] W. Curtis Preston: server.
But that, but that’s a lot more expensive.
[00:34:14] Stuart Liddle: If yeah, but, but if, yes, I, um, but I don’t think there was different retentions on that. It was the same retention and we, if we needed to back it restore something quickly, we could do it from the local copy rather than the remotely replicated copy.
[00:34:31] W. Curtis Preston: Why would you ever restore it from the remotely replicated copy.
[00:34:35] Stuart Liddle: If you had a problem with your, you know, primary site going down and, uh, you know, the backup system being compromised somehow, then you would want do that..
[00:34:48] W. Curtis Preston: Okay.
[00:34:48] Stuart Liddle: See, this is a test. I know
[00:34:53] W. Curtis Preston: no, not a test. It’s just, I didn’t, uh, um, I didn’t, um, I wasn’t there, man. Isn’t dare man.
[00:35:02] Stuart Liddle: okay. Oh, you,
[00:35:03] W. Curtis Preston: Um, and I didn’t ever have to administer those NetBackup appliances in production,
[00:35:10] Prasanna Malaiyandi: so we talked about now you went from tape to sort of VTL to backup appliances. Did you ever have to worry about, like, what do you do with applications that are moving to the cloud while you were there at Amgen?
[00:35:25] Stuart Liddle: Yeah, I did. And unfortunately I was not one of the main people that got into that. So I, my experience with the cloud based stuff is not as extensive as some of the other people that I worked with. So, um, yes, we did that. And, and in fact, there’s a lot of applications that ended up at, uh, AWS and, you know, things like in fact, um, one of my coworkers ended up going to work there.
So. Um,
[00:35:56] Prasanna Malaiyandi: Gotcha.
[00:35:58] W. Curtis Preston: And, and what was the, what was the general feeling of that? Was it, was it because you might have been in some of those, was it like saying, Hey, well, we definitely, well, we definitely gotta back that stuff up or, or was there a feeling of like, well, it’s the cloud, so we’re good.
[00:36:14] Stuart Liddle: no. I think the, um, the feeling was that that people thought that, that it was a good thing to use and they didn’t really take into account the full expense of that when they were talking about it, they were just looking at it from the standpoint of, Hey, we can reduce our data center footprint if we put the stuff to the cloud and virtualize it and not necessarily realizing that in some cases you’re gonna be spending as much if not more on stuff like that.
So,
[00:36:48] W. Curtis Preston: Yeah, subscription based pricing for hardware is essentially what that is
the
[00:36:53] Stuart Liddle: exactly.
[00:36:56] Prasanna Malaiyandi: So going back to an earlier conversation we had in the podcast where the, uh, person was afraid of their job getting sort of obsolete.
Were you afraid of that when they were starting to move to the cloud? And were you thinking, oh man, is this going to mean I’m out of a business or out of a
[00:37:14] Stuart Liddle: no, no, not at all. And I think it primarily was because of the fact that there was still applications that they wanted to keep in house on prem and, uh, they didn’t want to, uh, put that out to the cloud. So no, there was still that. And then there’s also, again, that, that cost factor, I think, because if, if we got involved with it properly, we could point out that, Hey, this is still cheaper, you know, but, but I think that’s changing.
I haven’t. I haven’t been involved so much with it recently. You guys probably know more about that,
[00:37:57] W. Curtis Preston: At what’s changing.
[00:37:59] Stuart Liddle: um, the, the pricing for, for virtualization and cloud based stuff,
[00:38:07] W. Curtis Preston: I, I don’t know.
[00:38:10] Prasanna Malaiyandi: I would say you now have more options. Like you can pick cheaper tiers of storage.
[00:38:16] Stuart Liddle: mm-hmm
[00:38:17] Prasanna Malaiyandi: you can try to optimize and spin down, like instances when they’re not in use or allow them to be preempted. There is Lambda functions, which means you don’t necessarily run a server. You’re just running code when it needs to run.
So they’ve given you more options. I don’t necessarily know if it’s cheaper. right. But it’s
now up to the developer to optimize for those costs.
[00:38:41] W. Curtis Preston: you think Prasanna? Well, it, yeah, I. I think it is cheaper. If you use the cloud, the way cloud can be used. Like you can’t do it, you can’t do a Lambda function in a data center. The concept of running a Lambda function. is that you, you get to do this for next to nothing. And if it’s something that you do infrequently, or there, there, there is a crossover point, right, where you’re paying by the Lambda function, but it’s super, super cheap, but at some point you cross a boundary and now, well, this would be cheaper if I would just rent a VM for a certain number of hours per day.
Um, and so I, I think that the, the people that continue to struggle with cloud costs are people that don’t, they, they don’t treat the cloud like that. They, they just, they just treat the cloud as another data center. And so they, they say, well, we have a thousand VMs over here. We got a thousand VMs over there.
We got 20 terabytes of storage over here. We got 20 terabytes of storage over there, and that is the guaranteed way to have your costs skyrocket.
Doesn’t matter what the cloud vendor is. If you’re renting your hardware, cuz that’s what you’re doing, right. You’re renting your hardware. And, and I know I make this analogy a lot, but it’s like going from owning a car and you go, gee, I don’t want, I don’t want the troubles of owning a car anymore.
So I’m going to rent a car from Avis all the time. right. I, I, I don’t want to do, I don’t wanna do oil changes. I want that to be Avis’ problem. So I’m gonna rent a car from Avis 24 7 and have it sitting in my driveway. But I’m gonna, but, but renting a car is an incredibly expensive way to have a car. It’s great if you need it for a day or even a week, or you need a car where you happen to be, which isn’t where you live, that’s what Avis is for. And that’s the way the cloud should be. You should buy little pieces. And if what you’re doing is just running a, a server 24 by seven, then it’s going to be more expensive than having that same exact server sitting in your site.
Um, but I’m sorry, I jumped up on a soapbox there,
but
[00:41:07] Prasanna Malaiyandi: take your analogy one step further and say
that Lambda is the equivalent of Uber. In your example.
[00:41:14] W. Curtis Preston: Agreed. Lambda is the equivalent of Uber and Uber is a great cheap way to do, to do certain things, right. But if you take Uber all day, Is it gonna be even more expensive than renting that car? Right? Shoot, actually, you don’t even have to take Uber that much. you know, you could take, take, take couple Ubers a day and you’re gonna start to be wondering, you know, if you should have a rental car, uh, is just, you know, again, this is where the analogy falls apart.
The only advantage of the Uber in that scenario is you can get drunk and not worry about it, but I don’t think the Lambda function, you know, is gonna help with that. Um, so, okay. Um, Stuart, is there anything that you, you know, like if you had had your druthers, if you could have waved a magic wand, is there anything that you would’ve wanted to have, you know, from a backup and recovery perspective that, that you didn’t get when you were doing things back in the day?
[00:42:18] Stuart Liddle: oh, geez. What was you mean beyond the Nirvana of site to site
[00:42:23] W. Curtis Preston: Yes. Beyond the Nirvana of site to site replication.
[00:42:27] Stuart Liddle: Uh, wow. Oh, well, okay. Um, full automation of the, uh, the process of, of upgrading. I mean, upgrading from one version to of. NetBackup to another, or, you know, having clients updated when you have to deal with different, um, uh, support groups, you know, like windows and Unix and VMware, you know, by, by, by, by telling you know, somebody, Hey, you best have your systems all on the latest version of NetBackup or you’re gonna have problems.
And they look at you and say, oh, you know, that’s a project, so we’re gonna have to take some time with that. And then just like throw up your hands and yeah, I wish I could have had that kind of thing. That would’ve been great.
[00:43:25] W. Curtis Preston: Yeah. And, and that is a real challenge for, because no one wants to upgrade their backup server. Right. And, and your clients, they don’t wanna upgrade their, their backup client because it’s just, it’s just, they don’t, they don’t really get any benefit out of it. Right.
[00:43:40] Stuart Liddle: well, they don’t get any perceived benefit out
of it,
[00:43:43] W. Curtis Preston: benefit out of it,
[00:43:44] Stuart Liddle: right?
Until something breaks.
[00:43:49] Prasanna Malaiyandi: then they’re like, why didn’t you
[00:43:51] W. Curtis Preston: So that,
[00:43:52] Stuart Liddle: how come you didn’t know about this?
[00:43:54] W. Curtis Preston: solving all the tape problem and everything that this was what you were left with, was that maintenance, that ongoing maintenance.
[00:44:00] Stuart Liddle: That ongoing maintenance and, and upgrading the backup servers was not a big deal because it was just our group that was, you know, involved with it. So us doing it was not a big deal, but to tell other people in other groups with different managers that, Hey, you’ve got a thousand Unix servers out there that need to be upgraded to a newer version of NetBackup.
They’re like, man, , we’ll just let it go.
[00:44:28] W. Curtis Preston: that, and, and again, I, I swear I did not give you this question just so I could say this, but this is one of the beauties of SaaS based backup is that you don’t have that problem. Right. You’re never upgrading the backup server and, and we even solved the, the, the updating of the client that, that could be done by the software itself. And you can, you know, schedule it I’d, I’d like it to just be completely automatic.
But I think that a lot of companies don’t want that. Right. They want some control over when that happens so that they know that it’s happening, but you wouldn’t, you wouldn’t have had to, um, you wouldn’t have had to spend a lot of time upgrading the servers and upgrading the clients. Cuz magic just would’ve happened.
But um, but Hey, but now you’re gone.
This is, this is a secret. This is a secret to backup. Success is leaving the job all together.
[00:45:17] Stuart Liddle: Now, now I’m gone, but let me just put in a shameless plug and say to anybody out there, who’s listening. If you’re interested in hiring me as a NetBackup admin,
[00:45:29] W. Curtis Preston: Absolutely
[00:45:30] Stuart Liddle: me.
[00:45:32] W. Curtis Preston: Stu Stuart knows his stuff and he’s got grandkids to pay for.
[00:45:35] Stuart Liddle: yeah. That’s right.
[00:45:37] W. Curtis Preston: How many, how many grandkids you got going on there?
[00:45:40] Stuart Liddle: three, so far
[00:45:42] W. Curtis Preston: Three so far.
[00:45:43] Stuart Liddle: but two, two here in Denver and one in Seattle.
[00:45:47] W. Curtis Preston: Yeah. Cuz you now live in Denver.
You you’ve been following your, you’ve been following your grandkids around.
[00:45:53] Stuart Liddle: Yep.
[00:45:54] W. Curtis Preston: Yeah. Yeah. That is that. That may be a future. That may be my future as well. Uh, right now my grandkid,
uh, I, my office is their former bedroom and their other bedroom is now right above me. So I don’t have to travel anywhere to see my grandchild.
Stuart, so Stuart, I wanna say, you know, thanks for coming on. I mean, it’s been great to,
to, you
know, we talk about you so many times, your name has come up on this podcast.
So many times as like the guy that told me to put a testing. Chapter into my book and all of that. Right. Um,
[00:46:32] Prasanna Malaiyandi: Yeah, your name does come up quite often. Stuart and it’s great to finally meet you.
[00:46:37] W. Curtis Preston: and not, and not just because, you know, we like to giggle at your name
[00:46:41] Stuart Liddle: Yeah. well, nice.
[00:46:44] W. Curtis Preston: for the, for those listening and not reading, uh, it is Liddle with, with Ds, uh, Stuart Liddle, not little, uh, he’s not, he’s not the mouse, but, uh, anyway, so, and, uh, and,
[00:46:58] Stuart Liddle: Prasanna’s not a cat.
[00:47:00] W. Curtis Preston: Prasanna’s not a cat, uh, and Prasanna, thanks your questions as well.
[00:47:05] Prasanna Malaiyandi: I should say this was a pleasure and I hope I wasn’t poking too much fun at you Curtis and asking Stuart to, uh, reveal all the mysteries about Curtis
[00:47:14] W. Curtis Preston: No one, no one knows all the secrets. Um, and, uh, I want to thank you again to our listeners. Remember to subscribe so that you can restore it all.
The post Backup practitioner tells stories from the trenches appeared first on Backup Central.