Work Screw ups???

[wayniac]

So, last night I as im at back to school night, I get a call from an engineer that a project I am working on has an issue and the parts built are now trash. Turns out there was a pretty major screwed up. The loss is over 75k, and a few days time. Its bad enough im a little worried about going into the office today if my key card will work. I have been there 14 years, and this is the first issue to cost us money. I just hope upper management isnt looking for someones head to roll for this one. I believe im a great employee but you never know.

So im curious what other people have done... I know someone here has screwed up bigger than this to make me feel better...

Wayne

 

[Done-it-again]

What do you do with your company?

 

[wayniac]

What do you do with your company?

I am a circuit board designer.. We talk sometime that we are surprised that any of them work there are so many things going into these projects...

 

[Sleek-Jet]

Nothing I'll admit in public...

 

[wzuber]

I did a $20k job that cost me $30k when all was said and done. that was fun.

 

[Sleek-Jet]

I'm going to guess there is a QC process where everyone signs off on the design and process. So unless you dummied something to cover up the error; it might be an unhappy day at work but you won't get fired.

The nice thing about process controls is there is enough blame to go around.

 

[ChumpChange]

I had a coworker not kick the boxes on an inventory walk and many of the boxes were empty. Took a $12mil loss.

 

[ltbaney1]

I find it best to own it. Don’t try to hide it. I’ve been lucky and not had one at that cost. But my last one was about 5k, walked into the bosses office and just laid it all out on the table. Talked about what could be learned and done to prevent it in the future. Took my lumps and moved on.

 

[LakeBeard]

We bid and sold supports for 8k each. Each support COST us 64k. (14 total)

 

[Bear Down]

I guess it depends on what your profitability is. I'm in sales and export tons globally, but there is always the few bad arrivals that hurt badly, especially with air freight being so expensive. My most expensive was about $12,000 on a order to the Middle East, but my profit on my portfolio for my company is about $125,000k per month. These don't happen often, but when it does, it just hurts, but I don't worry about it much, because I have 51 other weeks in the year to make it up. Company gives me tons of freedom and trusts me with their money.

 

[Bear Down]

I had a coworker not kick the boxes on an inventory walk and many of the boxes were empty. Took a $12mil loss.

You win..

 

[Uncle Dave]

So, last night I as im at back to school night, I get a call from an engineer that a project I am working on has an issue and the parts built are now trash. Turns out there was a pretty major screwed up. The loss is over 75k, and a few days time. Its bad enough im a little worried about going into the office today if my key card will work. I have been there 14 years, and this is the first issue to cost us money. I just hope upper management isnt looking for someones head to roll for this one. I believe im a great employee but you never know.

So im curious what other people have done... I know someone here has screwed up bigger than this to make me feel better...

Wayne

We in house design and make (CM out) many boards for several applications.
It happens all the the time on initial yield testing on a new board so we keep the run small as possible.
Fully 1/3 of the time we need to do a board spin anyway and the initial yield becomes effectively trash

If it's the first time in 14 years thats a pretty good track record.

Someone should have checked your work before committing to a run so save some hate and or guilt for the process you have in place.


UD

 

[SoCalDave]

I had a coworker not kick the boxes on an inventory walk and many of the boxes were empty. Took a $12mil loss.

 

[SoCalDave]

Got screwed once on having some parts quoted for repair. Quote was WAY HIGH so told him I wanted the parts back. He told me it will be $3800 for the inspection when all they did was pressure wash the parts. Learned my lesson on that deal.

 

[wayniac]

I'm going to guess there is a QC process where everyone signs off on the design and process. So unless you dummied something to cover up the error; it might be an unhappy day at work but you won't get fired.

The nice thing about process controls is there is enough blame to go around.

yes, definitely some other people that signed off on this... our director tends to fly off the handle sometimes, and upper management makes these decisions from a different area...

 

[wayniac]

I had a coworker not kick the boxes on an inventory walk and many of the boxes were empty. Took a $12mil loss.

That is insane... i knew someone had a story...

omg

We in house design and make (CM out) many boards for several applications.
It happens all the the time on initial yield testing on a new board so we keep the run small as possible.
Fully 1/3 of the time we need to do a board spin anyway and the initial yield becomes effectively trash

If it's the first time in 14 years thats a pretty good track record.

Someone should have checked your work before committing to a run so save some hate and or guilt for the process you have in place.

UD

with customer lead times so short, management has decided that full runs is another way to save time. Designs have been getting harder, and times shorter... There were others that checked this one, but I will be the one meeting with the boss today. Guess its time to head in and get my beating and see if my keycard works...

 

[HBCraig]

Been involved in a few. I wont say where or who, but a recent stadium build that one of my cranes is on had a bad one. The upper ring at the top of the stadium is fabricated in big sections on the ground then flown into place by the cranes. This piece was about 800k lbs and was off by a good chunk. Over a foot from what I was told. That steel mishap must have been big

 

[pronstar]

I had a coworker not kick the boxes on an inventory walk and many of the boxes were empty. Took a $12mil loss.

Damn...Were they boxes of cash?


Sent from my iPad using Tapatalk

 

[CLdrinker]

I’m sure your boss will give you 2 weeks notice before letting you go.

 

[monkeyswrench]

I can't nearly come close on dollar amount. If we based on percentage though...
I was doing an elastomeric roof coating job. This was in 2011, times were tight, running a crew of three. The three were myself, a "hose bitch"...the guy who makes sure you don't wad up on the hose, and a "barrel monkey"...the guy that makes sure you don't run out of paint. Well, the perfect storm occurred. The airless pump was about a 20hp deal, every cycle rubbing on the parapet wall. We always had a packing blanket to protect the hose from chaffing. It had always worked before...
Long story somewhat short, 3000psi turns into a hell of a sprinkler. I had notice the spray pattern changing, but it was close to time to swap barrels, so didn't think much of it. When the barrel monkey saw it, he jumped on the truck to shut the rig down. In his haste, he knocked an empty off the truck. Empties are never empty, and the drop tarp at the back of the truck wasn't there for containment, just splatter.
Like I said, it was 2011. To get the job I had bid it T&M, just materials and days wages... By the time the property was cleaned up, the guys were paid and about 10 cars were detailed, it cost me about 6k.

 

[ChumpChange]

That is insane... i knew someone had a story...

omg


with customer lead times so short, management has decided that full runs is another way to save time. Designs have been getting harder, and times shorter... There were others that checked this one, but I will be the one meeting with the boss today. Guess its time to head in and get my beating and see if my keycard works...

That’s not the biggest I know of either. I know an international transaction that went the same way for some $60mil. Banking is all about risk and sometimes the risk wins.

Damn...Were they boxes of cash?


Sent from my iPad using Tapatalk

They were boxes of air.

 

[farmo83]

I was working as an accountant at the time and accidently wired one of our counterparties 40 million dollars twice. They returned everything but it took time. I claimed I needed more training on the payment system process which was sort of true at the time. I got put on lay off notice about 6 months later and I’m sure this little incident was a huge factor in how I got picked for that.

 

[Sleek-Jet]

Damn...Were they boxes of cash?


Sent from my iPad using Tapatalk

Close; wood shutters.

 

[Xtrmwakeboarder]

A couple years ago I jumped into a new Controller role, was drowning in work, and didn’t take a step back to look at some of the big picture items and just kept rolling with SALY. “Same as last year” About 6 months in, I’m approving invoices and notice no sales tax... strange, these aren’t for resale... Talk to the group and CFO immediately and they have no idea why we aren’t charging, and have never charged. I do an analysis and come up with a liability of over $3m. Fuck. I engaged some specialist, lawyers, etc. and eventually found a loophole, but that wasn’t just scary from a cashflow perspective, but also possibly restating financials of a public co. The key to these things is assessing the damage, coming up with a possible fix, implementing controls so it doesn’t happen again, and keeping mgmt. in the loop with your plan.

Edit: And no, prior mgmt. did not know about the loophole, we just got lucky.

 

[bocco]

I am a circuit board designer.. We talk sometime that we are surprised that any of them work there are so many things going into these projects...

View attachment 792060

I design circuit boards as well. I'm curious to the details of what screwed up. Was all of the input from the engineer correct? Was the schematic input from the engineer or did you do that based on sketches from the engineer.

We got bit recently when a power supply chip had the pin ordering opposite the standard for BGA packages.

 

[wayniac]

well I am still employed. The director did go off ranting.. talking about heads rolling etc... still a little uncomfortable around here, but we are fixing the issue and adding to the final release checks.. I hate being on the email chains with the big bosses, but I learned that we had built in buffer time...

I design circuit boards as well. I'm curious to the details of what screwed up. Was all of the input from the engineer correct? Was the schematic input from the engineer or did you do that based on sketches from the engineer.

We got bit recently when a power supply chip had the pin ordering opposite the standard for BGA packages.

bocco, Not very often i talk to anyone that knows what I do...

the issue was a shield plane not being tied to GND. This board has 25 power plane 2 independent Ground planes and 5 individual ground shield layers. The plane layer looks normal just not tied to ground. ipc checks and connectivity checks are not an issue because gnd is connected and since its no net, it doesn't show up as floating, and doesnt short to anything. We are thinking maybe valor or something for future checks.

That is a bummer about the pinout. makes me have flashbacks of sot23 packages when i first got into this industry. How long have you been doing this?

Wayne

 

[$hot]

Congrats on your new position... Under the bosses thumb

 

[monkeyswrench]

Congrats on your new position... Under the bosses thumb

At least it's not under the bosses desk

In reading these, it's interesting to see people in much more respectable industries still have "Oops's".

At the end of the day, you can still come home, have a roof over your head and food in the kitchen. That's all that matters!

 

[bocco]

well I am still employed. The director did go off ranting.. talking about heads rolling etc... still a little uncomfortable around here, but we are fixing the issue and adding to the final release checks.. I hate being on the email chains with the big bosses, but I learned that we had built in buffer time...



bocco, Not very often i talk to anyone that knows what I do...

the issue was a shield plane not being tied to GND. This board has 25 power plane 2 independent Ground planes and 5 individual ground shield layers. The plane layer looks normal just not tied to ground. ipc checks and connectivity checks are not an issue because gnd is connected and since its no net, it doesn't show up as floating, and doesnt short to anything. We are thinking maybe valor or something for future checks.

That is a bummer about the pinout. makes me have flashbacks of sot23 packages when i first got into this industry. How long have you been doing this?

Wayne

Wayne,

I have been in the business since 1976. Started as an Electro-mechanical drafter and got full time PCB around 1984. Yeah, a lot of people have no idea what this is. Not sure if Valor would pick up your problem. I think the problem is at the schematic level and the shield plane needs a net name of some sort. Either call it GND and connect with just a couple of vias or give it another name and force a short somehow. We use Xpedition sftware and it now allows intentional shorting at vias.

When you have time check out the spec sheet for a LINEAR TECH LTM4634IY. Look at the BGA pin out. It's opposite every other BGA out there.

Garyy

 

[wayniac]

At least it's not under the bosses desk

In reading these, it's interesting to see people in much more respectable industries still have "Oops's".

At the end of the day, you can still come home, have a roof over your head and food in the kitchen. That's all that matters!

agreed about what matter, but it makes you think real fast.. we have emergency money saved, and would be fine for a while, but i would hate to spend it... Its funny the different perspective on what people find respectable. I sit in dark rooms and work strange hours. Every project I work on can cost my company millions in revenue if I screw up... Stress level is high...

 

[Bigbore500r]

I had a coworker not kick the boxes on an inventory walk and many of the boxes were empty. Took a $12mil loss.

He was the guy signing the flooring checks to Magic back in '07 ?

 

[Bigbore500r]

So, last night I as im at back to school night, I get a call from an engineer that a project I am working on has an issue and the parts built are now trash. Turns out there was a pretty major screwed up. The loss is over 75k, and a few days time. Its bad enough im a little worried about going into the office today if my key card will work. I have been there 14 years, and this is the first issue to cost us money. I just hope upper management isnt looking for someones head to roll for this one. I believe im a great employee but you never know.

So im curious what other people have done... I know someone here has screwed up bigger than this to make me feel better...

Wayne

If you have a proven track record of performance and no history of repeat performance issues, it would be very foolish of the company to terminate you on these grounds. Best of luck, hope you get it ironed out and put it behind you

 

[ChumpChange]

He was the guy signing the flooring checks to Magic back in '07 ?

lol. That was the difference in financing terms between a manufacturing company and a dealership. Different advance rates for each industry......IF THEY'RE NOT RELATED!!!!

 

[Deckin Around]

Damn...Were they boxes of cash?


Sent from my iPad using Tapatalk

Kilos

 

[monkeyswrench]

agreed about what matter, but it makes you think real fast.. we have emergency money saved, and would be fine for a while, but i would hate to spend it... Its funny the different perspective on what people find respectable. I sit in dark rooms and work strange hours. Every project I work on can cost my company millions in revenue if I screw up... Stress level is high...

I define respectable as all those jobs my parents wanted me to have. Collared shirt and an income not dependent on the weather. As I've become older, I look at it this way, one form of employment is harder on your exterior parts, the other on your interior. I learned that when I started being more the owner, and less the monkey with a hammer. I'd much rather be the monkey with the hammer. At least then you know why you hurt, and that's why you can't sleep. On the office side of life, often you find yourself awake at night wondering why you're thinking of five thousand things, but maybe only one or two matter. After a few years, I figured out what was important to me: My family...if they had a home, food and health was decent, not a whole lot else mattered.

 

[bocco]

agreed about what matter, but it makes you think real fast.. we have emergency money saved, and would be fine for a while, but i would hate to spend it... Its funny the different perspective on what people find respectable. I sit in dark rooms and work strange hours. Every project I work on can cost my company millions in revenue if I screw up... Stress level is high...

It sounds like you might want to start looking around for a different job. I think I'm pretty well set where I'm at. I'm hoping to ride it for 5 more years and then ride off into the sunset. Or if my wife has her way I will ride off into the sunrise. Either way I still watch the job market in case a unicorn shows up.

 

[rrrr]

In 1976, I was 19, and working as a superintendent for a medium sized general contractor. The company had about 20 employees, and was owned by a friend's father, Gerry. The company built commercial and multifamily projects.

My friend's cousin Mike was also a superintendent, in his case nepotism being more important than skill or knowledge. The GC was the successful bidder on a three building condo complex in Vail, and Mikey was sent to oversee the project.

The buildings were three story wood framed structures shaped like a capital "C", and built along creek frontage. Site work had started as soon as the snow could be pushed away and no more major storms were expected. By the end of March, the underslab plumbing and electrical for the center building was completed, forms were built, and concrete pouring commenced.

The morning after the center building slab was poured, my friend's dad met the Denver based architect on site for a progress inspection. They both immediately noticed the multiple plumbing and conduit stubups located in what was going to be the landscaped front center of the "C" shape when construction was finished. That was odd, because the courtyard area only required a couple of area drains and a few pedestal light fixtures.

After consulting the drawings for a very short time, the reason for the dozens of stubups in an area where a courtyard was supposed to be was determined. Mikey had laid out the building lines 180° from the intended orientation. Instead of facing a parking area, the open side of the "C" was overlooking the creek.

Mike had no knowledge of the plumbing or electrical trades and didn't know anything about what would or wouldn't be in the slab. Since it took just one day to form the exterior perimeter of the slab and install the prefabricated rebar beams and the mat, no one had called out the error before the concrete trucks and pumps showed up and did their thing at 6:00 the next morning.

Mikey was fired after lunch. After much back and forth with the owner, GC, architect, and structural engineer, the concrete contractor overcut the unwanted slab in the courtyard area about 4', then carefully broke out another 3' to expose the rebar, and a 40 diameter rebar splice, plumbing, and electrical was tied in. The exterior perimeter beam areas were also excavated, and 3' of the poured beams were broken back so the beam rebar could be tied in, again with a 40 diameter splice.

As you might imagine, the remedial work was expensive. In fact, very expensive. It also impacted the construction schedule, and the buildings weren't dried in until after snowstorms had started in early November.

Mikey found employment elsewhere, and he didn't attend any family functions like Christmas for a long time.

I always wondered how the subcontractors could have allowed the error to happen. It seems crazy that weeks of work was done before the concrete was poured and the error discovered. It was likely because they just had the drawings for their trade, and never looked at the site plan. That was common, and probably is to this day.

 

[endobear]

I can't nearly come close on dollar amount. If we based on percentage though...
I was doing an elastomeric roof coating job. This was in 2011, times were tight, running a crew of three. The three were myself, a "hose bitch"...the guy who makes sure you don't wad up on the hose, and a "barrel monkey"...the guy that makes sure you don't run out of paint. Well, the perfect storm occurred. The airless pump was about a 20hp deal, every cycle rubbing on the parapet wall. We always had a packing blanket to protect the hose from chaffing. It had always worked before...
Long story somewhat short, 3000psi turns into a hell of a sprinkler. I had notice the spray pattern changing, but it was close to time to swap barrels, so didn't think much of it. When the barrel monkey saw it, he jumped on the truck to shut the rig down. In his haste, he knocked an empty off the truck. Empties are never empty, and the drop tarp at the back of the truck wasn't there for containment, just splatter.
Like I said, it was 2011. To get the job I had bid it T&M, just materials and days wages... By the time the property was cleaned up, the guys were paid and about 10 cars were detailed, it cost me about 6k.

Was doing an exterior warehouse job on a weekend by myself and ran over my airless hose with the lift on accident. Running 250' of hose off a big ass 5500 psi gas machine out of a 55 gallon drum in light blue. By the time the lift came down and I got to the truck to shut the machine down I had pumped about 20 gallons of DTM onto the new blacktop parking lot.

Buddy of mine in the painting business had a guy get oil based overspray on over 60 new cars spraying in 40 MPH wind at a car dealership. Was the painters 1st week on the job. Cost him his business.

 

[Nordie]

Been involved in a few. I wont say where or who, but a recent stadium build that one of my cranes is on had a bad one. The upper ring at the top of the stadium is fabricated in big sections on the ground then flown into place by the cranes. This piece was about 800k lbs and was off by a good chunk. Over a foot from what I was told. That steel mishap must have been big

I know I know....held the job up for quite a while too

 

[kiwi77]

Screw up work flow chart...

 

[WhatExit?]

I know a guy who lost thousands on a boob job that he never got to see, feel or break in.

Like many mistakes there was more to it than just the cost

 

[Your ad here]

I've had little stuff compared to some of you. Paved using the wrong asphalt. About 12 tons. Had to grind it out and repave. Ordered 40 tons of to much crushed base a few times. Broom finished a concrete planter when it was supposed to be a smooth finish. Poured an access ramp that wasn't in the contract but on the plans. Broomed it to early and had to rub brick it a week later. The Boss, a Chief, and an Indian were on a paving job. Chief told me (indian) to order the cleanup load. I did. Boss wasn't too happy, he stepped out the tonnage for the cleanup and wanted to order it. Ended up with 6 tons left over. The Chief and I kept our mouths shut the rest of the day.

 

[Big B Hova]

I'd start a go fund me page. It's what everyone does now

 

[Kachina26]

Was doing an exterior warehouse job on a weekend by myself and ran over my airless hose with the lift on accident. Running 250' of hose off a big ass 5500 psi gas machine out of a 55 gallon drum in light blue. By the time the lift came down and I got to the truck to shut the machine down I had pumped about 20 gallons of DTM onto the new blacktop parking lot.

Buddy of mine in the painting business had a guy get oil based overspray on over 60 new cars spraying in 40 MPH wind at a car dealership. Was the painters 1st week on the job. Cost him his business.

I worked at a dealership that had a shit-ton of cars oversprayed by a sign company. They're still in business, so I guess it wasn't your buddy.

 

[CarolynandBob]

Many years ago I bid a job wrong. At the end of the job we had lost 100K. Luckily, that customer saw and liked our quality and our company. We ended up making millions off of that company. I would joke for years that I planned it that way. Owner never believed me, but something to joke about.

 

[rickym20]

I work for a metal deck contractor that has been spreading its roots through the US. We just got a call from a jobsite we completed a few months ago in Salt Lake City that the next contractor after us(roofer) overloaded the metal deck and crushed the metal deck in a big portion of what we did. Rough estimate is about 300k . I feel for the foreman in charge of the roofers.

On the other side in my career I was the field foreman installing some decking over a chiller. The decking required to get welded to the structure ( I-beams) and our firewatch guy must have been sleeping or heard he was supposed to watch fire. Burned up all the insulation costing our company around 13k after the dust settled. I was sweating bullets for awhile. Since then we have changed our company policies on how we work around equipment.

 

[Nordie]

Just simple math, but yesterday I was short 3 #8 bars, shop could not count. I ordered them on Friday as I had a shortage somewhere else on the job. So the 3 bars (shop shortage) slipped through the cracks, but the 45 short did not. I get a delivery for the 45 and not the 3 bars. The 3 bars was pressing as the contractor wanted to pour concrete this morning. Anyhow they ended up fabricating the bars, and my guys had to stay on overtime to make the concrete pour for this morning. I figure between trucking, fabrication, and overtime those three bars cost well over 5 grand.

 

[was thatguy]

When we drill directional wells we have close to a million dollars of electronics downhole. The MWD (Monitor While Drilling) tool gives us all our data and guides us along with surveys and all kinds of other data like Gamma resistance, temp, borehole circ pressures, and a multitude of other stuff depending on what the client has contracted and the type of well we are Drilling.
The tool itself looks like a long brass rod that hangs off inside a non mag Drilling collar assembly. It requires programming at surface, and interfaces with computer programs at surface that decipher the data and puts it into a displayable format while Drilling.

It is not unheard of for the MWD hand to mess up his programming either with the tool or with his surface equivalent or both.
This gives false data. Depending on the error it can be seen and corrected usually before you get too far, but can require a trip out of the hole.
Depending on the operation, a round trip can cost the client $10K to $50K pretty easy. If our tool fails or is programmed wrong we eat a portion of that cost.
If we get stuck at anytime it’s also possible to lose the entire assembly downhole. The client has the option of insuring up to 50% of the cost of tool replacement for around $5K a day, but rarely do they choose to accept it.

I once forgot to have the client sign the waiver of acceptance and we lost the string downhole.
The big wigs hashed it out over a martini lunch and it ended up that they back paid the insurance for like 7 days ($10,500) and paid half the value.
That cost my then employer about a quarter million bucks. I got suspended for 90 days with out pay.
This was many years ago, and now most outfits have a series of “sign off” sheets that must be signed by the DD’s, client, and MWD hands before Drilling commences.
They have to be emailed to the bosses at the time of signing and approved before the bit touches dirt. Doesn’t matter if it’s 2:30 in the morning that proccess is followed.
The MWD programming is also verified in the same manner.
I’ve never done it, but we’ve had DDs and MWD hands program incorrectly and go the wrong direction and hit adjacent wells destroying both wells to the tune of nearly $10 million, plus lose an incalculable amount in lost resource production, as well as endanger the rig and crews due to hitting active producing wells.

So these days there is a mountain of verification sheets that all must sign off on in the field each and every time we start a run.

 

[Boozer]

If you own a business you’ve lost money at some point, there’s a good chance you’ve lost some aliens money at some point as well.

We’re human, we make mistakes. That’s just how it goes.


Sent from my iPhone using Tapatalk

 

[NicPaus]

Few weeks ago was out with 2 friends that work in the fuel industry. One of them made over a million dollar mistake the week before. Another buddy is his boss. He volunteered to piss test but said he did not make him and no write up. Must be nice to be making millions a day where employees can make a million dollar mistake and not even get written up. Some industries the profit margins are much better than expected.