What are some conceptualizations that work in mathematics but are not strictly true? I'm having an argument with someone who thinks it's never justified to teach something that's not strictly correct. I disagree: often, the pedagogically most efficient way to make progress is to iteratively learn and unlearn along the way. 
I'm looking for examples in mathematics (and possibly physics) where students are commonly taught something that's not strictly true, but works, at least in some restricted manner, and is a good way to understand a concept until one gets to a more advanced stage.
 A: Here's a technical one I ran across at the beginning of grad school which always stuck with me. During questions after a colloquium talk on condensed matter physics, the speaker mentioned the fact that defects in these systems could be classified by topology. In the process, he equated the notion of homotopy methods with the concept of winding numbers.
This raised my eyebrows, since in my last undergraduate semester I'd spent a good amount of time reading up on my own about homotopy v. homology in the context of Cauchy's theorem in complex analysis. The problem with conflating homotopy with winding number is that, at least as the term is used in complex analysis, winding numbers are additive (i.e. commutative) whereas the first homotopy group could perfectly well be nonabelian. (In other words, there's situations where contours are non-contractible yet still integrate to zero because the winding numbers sum to zero).
So if one interpreted winding numbers in the complex analysis way, the analogy of that with homotopy just doesn't work. That bothered me for quite a while, and at some level it still does: defects can be classified using homotopy groups, yes, but those groups are free to be nonabelian. (Now I justify that language to myself by noting that Grassman numbers aren't commutative either.)
A: I can think of two:
1) For teaching elementary calculus, treating $\dfrac{dy}{dx}$ as a fraction is a great way of working with things like the chain rule, separable differential equations etc.
2) "To multiply by $10$ add a zero to the end" works perfectly for integers, and fails spectacularly for decimals.
A: Physics and mathematics are vastly different about this.
Physics: everything is not entirely true
For example,
Newtonian mechanics is "true" and good enough when objects are not too big, not too small and not moving fast enough,
Relativistic mechanics is "true" and good enough when objects are not too small,
Quantum mechanics is "true" and good enough when objects are not too fast and too big,
and fringe cases keep cropping up in which none of our theories about the universe work, so we make up new ones and each theory has its scope of application.
So no, you are not lying or oversimplifying when you're explaining Newtonian mechanics in the same way you're not lying or oversimplifying when you speak about quantum mechanics, you simply need to be clear about a theory's area of application.
Mathematics: always tell the Truth, nothing but the Truth, but never the whole Truth
For example,
For every two points there is a unique straight line connecting them - it's true, but it's not the whole truth, 
the proper sentence starts with "In a euclidean space", 
and then it's the whole truth, but try explaining that in elementary school and you'll just confuse students from the issues that you're trying to explain (i.e. how to draw a perpendicular line).
A: The velocity addition:
The first semester of my study I've been taught that if a guy walks with veolicty $v_g$ in a train that has velocity $v_t$ then his velocity with comparison to someone out of the train is 
$$v_r=v_g+v_t.$$ During the second semester I have been taught (by the same teacher) that  $$v_r= \frac{v_g+v_t}{1+\frac{v_gv_t}{c^2}},$$ using the Lorentz transform.
EDIT:

@vonPetrushev: Can you elaborate? I assume the teacher explained that they're talking about two different theories, and, if it was a good course, that the first theory is a niche in the second and accounting only for the cases where v_g v_t << c^2.

First semester we did classical mechanic (Newton's mechanic). The second semester we got an introduction to the special relativity (including Lorentz transform, etc...). For sure the teacher pointed out that, in facts, Newton was wrong but his approximation is already very powerful. Except if the guy walking in the train a superhero (and runs VERY fast), the difference is negligible. This comment marked me because it was also showing that the models we are actually using (even the most complicated and complete ones) may still be wrong. The whole "art" is to see which theory will provide a result sufficiently precise for the expectations. It makes less sens to calculate the velocity of an apple falling on a head with special relativity tools, however these tools can be relevant when computing some spaceship trajectories, etc.
A: This is more of a collection than a single answer, but otherwise each of this would be too short for an answer:


*

*"convex function can hold water": I frequently heard this while tutoring maths as something people get taught in highschool. Easy counterexample would be $e^{x}$.

*"if a bounded function do not converge, it must oscillate": well there is precise mathematical meaning of oscillation, but it is never explained in calculus class. Most people when heard "oscillate" will think of something that go up, then down, then up, then down, indefinitely.

*"Turing machine is just like usual computer": I only heard it once, presumably due to some poor explanation in high school, from a freshman who don't understand why we can take array access to be constant time.

*"since the denominator is 0, the amplitude must increase to infinity": almost always what the teacher say while solving wave with driving force assuming the solution to be a stationary sinusoidal solution to deal with the resonance case. I have the misfortune to have taken 12 different courses that end up solving this at some point, so I should know.

*"the error surface is like a terrain and the algorithm is like a drop of water, so it will get to the local minima", "to find fixed point, keep applying $f$ until adjacent term are close together": I have heard this in computational classes from science department, but luckily not in numerical analysis class in maths department.

*"the gradient is the direction of fastest rate of increase": not strictly wrong by itself, but combining with the usual issue of high schooler being only taught about gradient by its formula (rather than its actual definition) and trouble occur when function without gradient come up.

*"if 2 space is homeomorphic we can turn one into another with just twisting, bending and stretching, but no tearing, cutting or gluing": disregard the issue of what "twisting" and other action could means in an arbitrary topological space, this is still wrong even on nice simple subspace of Euclidean space. For example, a trefoil knot and an unknot are still homeomorphic.

*"semiproduct is like direct product, except that the group being extended end up wrecking its helper in the process": I just find the image too funny to not include. Well strictly speaking whether this is strictly wrong or not depends on how it is going to get misinterpreted.

*"nilpotent group still incur cost whenever you want to swap, but the cost get smaller the more you try": while this might be a nice way to show why the concept of nilpotent is close to abelian, this probably is going to be a bad way to think about nilpotent group in general.

*"the pdf of this distribution is similar to that distribution when the parameter are sufficiently large, so we can just replace this with that for all practical purpose": commonly used argument, but this is a classic confusion on method of convergence (I'm not even sure if there is even a notion of "convergence in pdf").
A: "If $f'(a)>0$ then $f$ is increasing near $a$."
Counterexample: $f(x)=x+2x^2\sin\bigl({1\over x}\bigr)$ for $x\neq 0$, $f(0)=0$, $a=0$. We have $f'(0)=1$, but there are points arbitrarily close to $0$ with $f'(x)<0$. 
A: Not really about mathematics, but a case where a highly respected mathematician, computer scientist and author believes (as you do) that it's useful when we are "taught something that is not strictly true, but works, at least in some restricted manner, and is a good way to understand a concept until one gets to a more advanced stage".
Donald Knuth writes, in the preface to the TeXbook:
"Another noteworthy characteristic of this manual is that it doesn't
always tell the truth. When certain concepts of TeX are introduced
informally, general rules will be stated; afterwards you will find that the
rules aren't strictly true. In general, the later chapters contain more
reliable information than the earlier ones do. The author feels that this
technique of deliberate lying will actually make it easier for you to
learn the ideas. Once you understand a simple but false rule, it will not
be hard to supplement that rule with its exceptions."
I heartily agree with this approach. In my view, many mathematics and
computer science texts spend far too much time fretting about odd-ball corner cases that do little to elucidate the general concepts. I find it annoying and unhelpful, personally.
A: Usually you are taught that topological spaces provide a nice framework for doing topology. But when you really sit down and try to prove some of the "obvious homeomorphisms" (for example $|X \times Y| \cong |X| \times |Y|$ for simplicial sets $X,Y$ with geometric realizations $|X|,|Y|$, or the exponential law $C(X \times Y,Z) \cong C(X,C(Y,Z))$), you will see that you will need more sophisticated models, for example compactly generated weak hausdorff spaces. See  here for more details. In my impression many lectures gloss over this deficiency of $\mathsf{Top}$.
A: I learned that the parabolic motion of projectiles is "wrong" in the short physics course I took at university. The path is not parabolic, it's elliptic, as the object that you throw is put in orbit the Earth until it hits the ground a few meters away.
Of course the parabole works perfectly well for throwing stones, since the stone never gets far enough to experience a pull along the x axis, but maybe you need to work with ellipses for throwing passive projectiles longer (though I don't know who uses trebuchets these days).
A: Geometry, and differential geometry, are full of these kinds of shortcuts, for the simple reason that geometry is a very visual, accessible, and venerable topic, resting on a deceptively deep body of theory required to make it rigorous. You are taught concepts in high school that will require many years of topology, linear algebra, measure theory, real and functional analysis, etc. before you have the full story.
Some examples:


*

*You are taught to calculate the volume and surface areas of solids by two kinds of slicing. Rarely is it explained why surface area requires approximating a surface with slices that are generalized cones, while generalized cylinders are good enough for volume, and I've never seen discussed why and for which kinds of surfaces these techniques don't work at all. More generally, it is often assumed that if you want to measure something about a smooth surface, you can do so by looking at a limiting sequence of discrete surfaces that converge to the smooth surface under "reasonable" refinement. Defining "reasonable" requires great care (beyond the scope of high school geometry) to avoid counterexamples like the Schwarz lantern.

*Concepts like arc length, surface area, curvature, etc. are not defined in a complete or rigorous way. On the one hand you have some formulas that clearly require a certainly level of differentiability, and on the other you have no problem calculating e.g. the surface area of a polyhedron, and probably don't bat an eye at the fact that polyhedra satisfy a meaningful version of Gauss-Bonnet, despite ostensible missing two degrees of differentiability! In addition to regularity, subtleties about orientability, compactness, etc. are likely not discussed. 

*Many theorems in plane geometry require objects to be in "general position" (usually left vaguely defined) and the corner cases are ignored. Similarly, pathological counterexamples to the formula for the Euler characteristics of polyhedra (e.g. polyhedra with odd Euler characteristic) are usually ignored.

*Finally, while not the same as teaching students incorrect facts, many geometric arguments you see in early classes implicitly rely on unproven "obvious" results like the Jordan curve theorem, invariance of domain, Sard's lemma, existence of triangulations, etc. that are far from obvious.
A: How about this notorious one I remember from high school?
"$f(x)$ is just a fancy name for $y$."
A: This is more like a long comment.

I am having an argument with someone who thinks that it's never
  justified to teach something that is not strictly correct. I disagree:
  often, the pedagogically most efficient way to make progress is to
  iteratively learn and unlearn along the way.

In math, or in science?
If we're talking about science then everything we teach is (at some level) incorrect. For example, Newtons laws break down at high energies; they predict the non-existence of certain observed phenomena, etc. So obviously, it is okay when teaching science to say things that aren't strictly correct.
However, if we're talking about mathematics, I think a strong argument can be made that it is never okay to teach things that we know to be incorrect.
Definition. Let $\varphi$ denote a mathematical theorem. Let us say that $\varphi$ is known to be incorrect iff there is a consensus on how to formalize $\varphi$, and that formalization is known to be false.
For example, here are some examples of theorems that are known to be incorrect.


*

*No continuous function $f : \mathbb{R} \rightarrow \mathbb{R}^2$ fills the plane

*Every function $f:\mathbb{R} \rightarrow \mathbb{R}$ is analytic and/or has a Fourier transform

*The 2-sphere cannot be turned inside out without any creases (see e.g. Smale's paradox).


In my opinion, it would be inappropriate to teach any of the above "theorems."
A: *

*A vector is just an arrow in space or an object which components transform in some particular way under rotations (see the question I asked in the Physics site).
In a similar fashion, a tensor is just a matrix.

*The fundamental theorem of calculus is the definition of a definite integral; it works, but it's wrong:
$$\int_a^b f(x)dx=F(b)-F(a)$$


*

*Given a primitive $F$ of a function, to get the set of all primitives we just have to add a constant, when in fact, it can be a picewise constant function (if we integrate a discontinuous function). 

*When solving differential equations, the assumption that the derivate of a convergent series converges too.

*Analytic is the same as differentiable/holomorphic (true in $\mathbb{C}$ but not in  $\mathbb{R}$).

*In Quantum Field Theory the usual definition of path integral has no mathematical rigour at all.
A: Thinking of Dirac delta function as a function works reasonably well up to a certain point. For example, every physicist knows that 
$$\int_{-\infty}^{\infty}e^{i\omega x}dx=2\pi\cdot\delta(\omega),$$
but only a small part of them really studied the theory of distributions.
A: In physics there is the point charge, and to a broader extent, any point particle, which is strictly an idealization.
The point charge is an idealized model. It is dimensionless and has infinite charge at the point. 
A: 
The shortest distance between two points is the unique line segment joining them.

This is not strictly true, since this line may even not exists, e.g. Taxicab geometry.

(Wikipedia picture)
A: Here are two false concepts taught in lower grades (explicitly or implicitly):
1) Every plane figure has an area. Elementary kids are not told that some figures have no area (or might not have an area, if you leave out the Axiom of Choice).
2) A set is a collection of objects satisfying any particular relation. (Some middle-school books avoid this by always discussing sets within some particular universal set, but this approach just caused me to raise questions even before I studied formal set theory.)
A: *

*Multiplication is repeated addition. So $\sqrt{2}\cdot \sqrt{8}=4$. Similarly division is repeated subtraction.

*Solve the differential equation $$\dfrac{dy}{dx}=y$$ Easy! Multiply both sides by $dx$, so $\dfrac{dy}{y}=dx$; now, integrating both sides, $\ln{y}=x+C$. 
I don't know how many people understand this multiplication by $dx$ (frankly, I don't). Even university students (at my university) take this step for granted. (Relevant: Is $\frac{dy}{dx}$ not a ratio?)

*The cancellation trick: $\require{cancel}x\cdot y= y\cdot z\implies x\cdot\cancel{y}=\cancel{y}\cdot z\implies x=z$. People use this 'trick' to prove $1=2$ etc. 

*High school teacher: Square root of negative number is not defined. So $x^2+1=0$ has no roots.
Secondary school teacher: define $\sqrt{-1}=i$ and $x^2+1=0$ has two roots $x=-i,+i$.

*"$1/0$ is $\infty$" Refer to point 5 here, this is correct.

*$\dfrac{\partial^2 f(x,y)}{\partial y \, \partial x}=\dfrac{\partial^2 f(x,y)}{\partial x \, \partial y}$. This can't be false in Electrodynamics.

*$1+\dfrac{1}{2}+\dfrac{1}{2^2}+\dfrac{1}{2^3}+\cdots =2$. The equality is little confusing. It should rather be considered as a limit of the sum (or limit of partial sums).

*$\sqrt{mn}=\sqrt{m}\cdot\sqrt{n}$. This is only true when at least one of $m$ and $n$ is positive. This 'trick' can be used to prove $1=\sqrt{1\cdot 1}=\sqrt{-1 \cdot -1}=\sqrt{-1}\cdot\sqrt{-1}=i^2=-1$.

A: I think the definition of a limit is a good example. In high school, I was taught that $\lim_{x \to a} f(x)$ is the value of $f(x)$ as $x$ gets closer and closer to $a$, or something equally nebulous like that. Similarly, $\lim_{x \to \infty} f(x)$ is the value of $f(x)$ as $x$ gets bigger and bigger. At the time, this was good enough; students in pre-calculus courses really don't need to understand limits much more rigorously than that.
Of course, later I learned that limits do in fact have a perfectly concrete definition:
$$
\begin{align}
\lim_{x \to a} f(x) = L &\iff \forall \epsilon>0,\ \exists \delta > 0\ \text{such that}\ 0 < |x - a| < \delta \implies |f(x) - L| < \epsilon \\
\lim_{x \to \infty} f(x) = L &\iff \forall \epsilon > 0,\ \exists c\ \text{such that}\ x > c \implies |f(x)-L|<\epsilon
\end{align}$$
I think it's clear that teaching most high school students these definitions would only confuse them.
A: We tell students in an introductory algebra class, say, that mathematicians invent axioms and then study the properties of the resulting object or theory. The story goes that some bright mathematician had the idea to write down the four group axioms before anyone knew anything about group theory, and then he and other mathematicians worked out all the known properties of groups from those four axioms.
In reality, it took mathematicians decades of working with concrete examples of groups -- permutation groups and groups arising from geometric symmetries -- to finally notice that all of those examples shared a few fundamental properties. These properties then became the axioms for the group, but only because the objects satisfying those axioms had already appeared naturally in other fields of mathematics. 
I think it is very rare for a mathematician to define some random set of axioms and then have something interesting to say about them. Instead, axiomatizing a theory is usually one of the final steps in making that theory rigorous. Axioms are more a sign of rigor mortis than of the potential for new results. Moreover, one theory -- most notably set theory -- can have competing axiomatizations which are all studied.
And yet, I think it is pedagogically useful to introduce students to the idea of an axiom in this way. A student who has studied no formal mathematics will not yet appreciate the subtle interactions between hypotheses, conclusions, and observations that govern mathematical progress. It is easier to give him or her a concrete set of "rules" to work with, and the cleanness of the ensuing theory -- by which I mean that steady march of (possibly unmotivated) definitions, lemmas, and proofs across the pages of math textbooks -- shows him or her our ideal for formal mathematics.
Edit: I realized that what I have written so far can be interpreted to mean something with which I vehemently disagree. Even though I think expositions of mathematics at the level of a first- or second-year undergraduate can benefit from an axiomatic treatment, any expositions that are more advanced -- certainly graduate textbooks, for instance -- should complement the rigid axiomatic method with thorough motivation for all definitions. The motivation is as important, if not more important, than the definitions and theorems!
A: I'm surprised nobody mentioned square roots before: I am 17 now and have always been told the square root of a negative number does not exist and can not be taken. I am interested in mathematics, so I know this to be false (though I don't really know what imaginary numbers are).
To not teach this at first makes sense, because when one first learns square roots one is generally not ready for imaginary numbers. (Until you are in high school IMHO - though the maths classes I take didn't and won't cover them.1)
1: where I live you can choose three types of maths: A, B and C, where A is mostly statistics and meant for people not very good at or interested in maths, C is a simplified version of A for people who are really bad at maths and B is 'real' mathematics, heavy on algebra and proving things, I think with B you'd learn about imaginary numbers. (As pointed out in the comments, if B isn't enough maths for you you can choose Maths D as a kind of add-on. Basically, choosing D means you get extra tough maths, on top of the regular B programme.) I had to choose A because otherwise I would not have been able to choose certain other subjects (though I was advised by my math teacher to take B, I chose these other subjects over challenging maths - my mistake).
A: Does the use of "wrong" definitions/formalizations count for your purposes?
For example, in high school it is probably reasonable (although not essential) to use the term "vector space" to mean "finite-dimensional vector space". This avoids the need to spend more time on the Axiom of Choice than is really called for by the curriculum, to justify the operation of taking a basis for your vector space.
The resulting mathematics might be correct in itself (in effect using an additional axiom in the definition of vector space), but the terminology contradicts standard.
More generally: foundations of mathematics are not taught early and I think your friend would have a hard time doing so. Pre-university education often appeals to naive set theory in a way that could easily be "abused" to demonstrate that the system in use is inconsistent.
Anyway, the fact that something is commonly done in a particular way, even if we can prove that it's pedagogically more efficient to do in that way, won't shift your friend's position that it's "unjustified" ;-)
A: Often in physics you pretend like higher order differentials vanish.  For example, you have $(x+dx)(x+dx)$, you multiply through and say that the $(dx)^2$ term is $0$.  You could try to appeal to nonstandard analysis for manipulating the differentials, but of course the amount of work involved to develop such a theory is hidden.  
A: Euclid's 5th postulate: two lines at right angles to the same line will never meet--and only if they are both at right angles. Well, in a perfect plane universe that might be true, but in the real world, non-Euclidean geometry is more "true": large triangles on the surface of the planet don't add up to 180 degrees, etc.
A: For the physics case, and I say this as a physicist not a mathematician having a dig:

Almost all of physics is "not strictly correct".

What I really mean is: "strictly correct" in mathematics means an entirely different thing to what it means in physics. In fact, in physics there is a much larger set of interpretations of the term than there is in mathematics. Mathematics as a discipline considers "correctness" as fundamental. Not so in physics.
People have different views as to how Physics works. Mostly it is about modelling the world such that experiments can be constructed that will show results consistent or inconsistent with the model.
What this means is that the mathematical correctness of the model is a completely different thing from its physical correctness.
Indeed, if you had a model known to be a bit flaky mathematically but able to be used to give experimentally valid results, then it is better than a total mathematically consistent one which does not.
Now you might say how can a "flaky" model not necessarily have a space of solutions that are non-physical and hence not experimentally valid. The answer is that those conditions may only appear in a situation we cannot actually observe or construct due to practical reasons.
So to come back to the claim almost all physics is "not strictly correct": there simply is not basis for ever saying that a theory is strictly correct. For example, the Lorentz velocity addition formula is "better" than the Newtonian one, but is a the end of the day a model applicable to specific situations only. We cannot for example, just go about adding velocities for bodies at different locations once we allow for General Relativity. 
It is in this sense that the current state of the art of physics "is a good way to understand a concept until one gets to a more advanced stage". Only if and when we have a serious candidate for the Theory Of Everything can (maybe) we go beyond this.
So to cap my answer, I'd suggest excluding physics from the question such that this becomes a non-answer . . .
A: When I was in primary school my teacher told me $\pi = 3$, then $\pi= 3.14$, then $\pi = \frac{22}{7}$, then $\pi$ was an irrational number and we could never write down $\pi$ completely. Then I was told  $\pi$ was actually transcendental.
But if I'd been told when I was 8 that $\pi$ was a transcendental constant I don't think I would have understood much about what it really represented to me at the time: the ratio of circumference to diameter of a circle.
A: In real analysis courses, students are often taught the equation x^2 = -1 does not have solutions before they learn what complex numbers are : i is a solution for this eqaution. 
A: The existence of Gabriel's Horn.

That is, there exists an object with a finite volume but with an infinite surface area.

Mathematically, we can show this to be true by taking a 360-degree revolution of the graph $y=\frac{1}{x}$ in the domain $[1,\infty)$ about the $x$-axis, and calculating its volume ($\pi$) and its surface area (infinite).
This is all well and good in theory, but in our physical world, no such object exists.
One of the reasons for this is that the thickness of the horn is limited by the size of the atom, so cannot be arbitrarily small.
A: How about e=mc^2?
This is taught as fact in almost every physics class, and is commonly 'known' to the public, but is actually a simplified version of the real formula.  You see, this says that the energy of an object is a product of it's mass and the speed of light squared.  This completely disregards the energy that an object can have due to its velocity (kinetic energy).
You could go even further into how the speed of light is taught as if it's a constant, when in reality the speed of light is variable based on the medium that it is travelling through.  If you ask a physics student what the speed of light is, they will rattle off a value without any indication that they are giving you the speed of light 'in a vaccuum'.  
I hope this helps!
A: What about all the basic rules of weight & motion--aren't they just simplifications of terribly complex rules that generally work as long as you don't deal with anything too small or going too fast?
It seems that EVERY problem in early physics/calculus is simplified to eliminate most of the variables because the problem would become impossibly complex if you added them?  For instance, falling object calculations don't generally take into consideration wind resistance, and if they do they don't take wind and varying pressure into account.  A ball rolling down a ramp always considers only perfect surfaces.  A draining tub doesn't take into account the speed difference of the funnel created by the flow or how long it will take to form?
Most things are simplified so we can fit them into our heads in one way or another.  I've always had lots of luck imagining electricity as water--I know it's inaccurate for many reasons, but it works extremely well even for things like induction (inertia), even though it's obviously "Wrong".
I also believe every description of what is going on in quantum mechanics is a best guess at this point, doesn't mean it isn't helpful though.
A: "There is such a thing as a set of goats."
