Mistake. If you're going to forth the effort, make it count. Go for the degree.

Ask any systems analyst with over 10 years experience. Put your intellectual effort into what will pay off the most in the long term. That's why we use and program for Windows (retch), despite its shortcomings, and why there are still COBOL (shudder) programmers, and why object oriented programming (shiver) is so pursued.

Take the math. Be very introspective about how you are doing. You need A's for what you want. If you aren't doing that well in a course, bail. Try again later.

Keep in mind that physics is a lot more fun than computers. Good luck, and remember that luck favors the well prepared.