Josh McDaniels: Raiders 'building' but it's slow, painful process play Stephen A. calls out Josh McDaniels after Raiders' loss to Colts (2:06) Stephen A. Smith details the Raiders' woes after their loss to the Colts. (2:06) 3:04 PM ET Paul GutierrezESPN Staff Writer Close Covered Oakland Raiders for CSNBayArea.com and Sacramento Bee for eight years Member of Pro Football Writers Association Previously worked at Los Angeles Times, Las Vegas Review-Journal and Sports Illustrated HENDERSON, Nev. -- Coach Josh McDaniels said he will stay the course amid rumors about his job security in his first year a day after a stunning 25-20 loss to the Indianapolis Colts that dropped the Las Vegas Raiders ' record to 2-7. "We are building," McDaniels said in his weekly Monday media conference. "I have never used the word 'rebuilding,' or anything like that. It's the National Football League; there's not five years to do that. "There's a process that we're trying to go through. Certainly, slower and more painful than anybody wants it to be." And while McDaniels said he had not been reassured of his status by Raiders owner Mark Davis in "some big clandestine meeting" in recent weeks, McDaniels said he will continue to try and build a foundation for the future. Editor's Picks Carr emotional at podium after Raiders' latest loss 15h Paul Gutierrez Colts winners in Saturday's debut, Ryan's return 16h Stephen Holder 2 Related "I'm going to control what I can control and do the best I can at my job and part of that is continuing to think about the future and next year and what goes on beyond the next few months here," McDaniels said. "If somebody tells me that my time is done, then my time is done." Expectations were high for Las Vegas after its unlikely run to a 10-7 finish and the playoffs last season under interim coach Rich Bisaccia and general manager Mike Mayock, after which Davis chose to bring in McDaniels and Dave Ziegler, as GM, from the New England Patriots . But the Raiders got off to a 0-3 start. And after winning two of three, they have lost their last three games in dispiriting fashion, getting shut out by the New Orleans Saints and former coach Dennis Allen, blowing a 17-0 lead (their third such loss of the season) at the Jacksonville Jaguars , and then Sunday's loss to the Colts, who had a coach who had never coached above the high school level in Jeff Saturday, an offensive coordinator who had never called plays in the NFL before in Parks Frazier and a 37-year-old quarterback who had been benched earlier this season in Matt Ryan . "We're not trying to do anything other than win every game and that's what we will continue to do," McDaniels said. "I think the reality is, I do understand the short-term history of this place. I was educated on it; they talked to me about it when I came here -- trying to get something together than can last and sustain and win and win and win and win and win. We're not doing enough of it now and we understand that. But at the same time, that was the vision for us, to try to figure out what is going to be part of the future, what isn't. And how we proceed going forward is going to impact that. "When you go through change, there's some things that you're going to see initially, and there's some things that you're going to see over time. When you have enough opportunity to make those evaluations and try to get it right eventually, that's the opportunity that you're hoping for." McDaniels said the goal is to "have a culture and the kind of people on the team that continue to try to uphold that year after year after year" in Las Vegas. "I understand the short-term frustration. I get it. I really do."