Is it possible to formalize the idea that large cardinal axioms don't limit us, while their negations do? This question has been substantially rewritten. Hopefully, its clearer now.
Suppose our goal is to describe every material set that can exist by extending ZFC. Of course, this is an impossible goal, since a consistent extension of ZFC cannot prove the existence of a model of its own axioms. However we can certainly make progress towards describing everything that can exist, especially by adjoining large cardinal axioms.
Now suppose that to ZFC we add "There exists an inaccessible uncountable cardinal." Intuitively, we haven't really limited ourselves in any way, with respect to the goal of describing every set that can exist. On other hand, suppose that to ZFC we add "There does not exist an inaccessible uncountable cardinal." Well, if our goal is to describe every material set that can exist, then I think adding this axiom is a very bad move.
In general, adding a large cardinal axiom $\phi$ to ZFC okay (so long as $\phi$ is consistent, etc.) while adding $\neg \phi$ is a bad move. Is it possible to formalize this idea, that large cardinal axioms don't limit us, while their negations do?
 A: One way we formalize this "limitation" idea is via interpretative power. John Steel describes this approach carefully in several places, so you may want to read what he says, in particular at

Solomon Feferman, Harvey M. Friedman, Penelope Maddy, and John R. Steel. Does mathematics need new axioms?, The Bulletin of Symbolic Logic, 6 (4), (2000), 401-446.

Briefly, the idea is that whatever framework we choose to formalize mathematics, should strive to accommodate "the broadest possible point of view". As Steel says,

The “broadest point of view” proviso is meant to exclude from attention the temporary adoption of restrictive assumptions as a convenient device for avoiding irrelevant structure. $\mathsf{V} = \mathsf{L}$ is often assumed temporarily for such reasons by set theorists who do not believe it, just as “all functions are $C^\infty$” is sometimes assumed by differential geometers who do not believe it.

If we adopt this approach, adopting large cardinals fits within standard heuristics such as Maddy's "maximize" that suggest that our foundations should be as strong as possible. The key here is that what we are maximizing is interpretative strength. This implies maximizing consistency strength, but the goal is interpretative power rather than the other way around. Adopting large cardinals provides us with a theory within which many other theories can be appropriately interpreted, even theories that do not mention large cardinals at all. A standard example: The theory "$\mathsf{ZFC}+$ All projective sets of reals are Lebesgue measurable" can be interpreted within the theory "$\mathsf{ZFC}+$ There is an inaccessible cardinal", but not within "$\mathsf{ZFC}+$ There are no inaccessibles". If we work within a strong theory, and it turns out to be convenient for some reason to adopt (temporarily) $\mathsf{V}=\mathsf{L}$, we can do this safely, as this theory is interpretable within ours by restricting all quantifiers in the relevant statements to range over $\mathsf{L}$. On the other hand, if we adopt "$\mathsf{V}=\mathsf{L}$", we cannot faithfully interpret theories that claim the existence of measurable cardinals, for example.
Steel discusses this asymmetry carefully: The instrumentalist dodge consists in adopting, say, $\mathsf{V}=\mathsf{L}$, together with all "low level" ($\Pi^0_1$, or arithmetic, or $\Sigma^1_2$, for example) consequences of, say, "$\mathsf{ZFC}+$ There is a measurable cardinal". More generally, given a theory $T$ and a set $\Gamma$ of sentences, Steel calls $\mathrm{Inst}(T,\Gamma)$ the theory "All theorems of $T$ in $\Gamma$ are true".

There are endless variations here. The theory $\mathrm{Inst}(T, \Gamma)$, used simply as a device to avoid directly asserting $T$ while retaining all its $\Gamma$-predictions, is a mathematical parallel of the physical theory: “There are no electrons, but mid-size objects behave as if there were.” That is, it is a parallel of this theory as it might be used by a philosopher of today, not as it might have been used by a physicist in 1900. Perhaps a cleaner parallel would be the theory that the world popped into existence 5 minutes ago, looking exactly as it would if there had been a past like the one we believe in. Mis-used in this way, $\mathrm{Inst}(T, \Gamma)$ is no more than an odd way of asserting $T$; it only becomes more than that if one has a program for ﬁnding a tool for making $\Gamma$ predictions which is better than $T$, and incompatible with $T$ in the realm of non-$\Gamma$ sentences. In evaluating a retreat from $T$ to $\mathrm{Inst}(T, \Gamma)$, one should  ask whether its proponent has any such competing tool in mind.

Steel's point is that there is not even a serious scenario (much less a developed tool) where assuming the negation of large cardinals (together, perhaps, with the low-level consequences of said large cardinals) ends up being advantageous in that it provides us with the means to interpret some mathematical theory that the adoption of large cardinals does not allow us to interpret.
There are a few issues here. One is that even if, theoretically, this makes sense, it may be that, in practice, the need for this all encompassing interpretability may not arise. However, there are examples where this has happened. The most prominent, that Steel mentions in his essay, is the study of regularity properties for low level projective sets, that could not really come to fruition until the theories of large cardinals and determinacy where sufficiently developed.

It doesn’t matter so much whether we can make do without this line of research; the more important question is whether we are better off with it. And it doesn't matter so much whether we call the subject which is better off (ordinary, core, normal) mathematics, or something else.

The other issue is that, perhaps, large cardinals are inconsistent after all, even if their low level consequences turn out to be consistent. There is an important point to make here: I expect that the development of inner model theory will take care of this issue if it actually arises. The tools we have developed to study large cardinals and their canonical inner models are sharp enough that we expect they'll end up uncovering inconsistencies (or mutually un-interpretable pairs of theories, or other anomalies). For example, we have developed tools to "compare" certain models via iterations. In all cases we can build these models, we can prove that the comparisons behave the way we want. In particular, this implies that theories of large cardinals have growing interpretability power matching up with their consistency strength. There are cases where we cannot yet build the models. What we do not have is cases where the models turn out to be incomparable. If we did, this would be significant. Our tools are designed in a way that I think this would be uncovered, if it actually happened. Instead, our tools keep allowing us to establish stronger and stronger positive comparability results. On the other hand, we should not discount that this optimistic reading of our partial results is what Hamkins calls "confirmation bias error", see here and here, but I think that there is really no evidence to expect the opposite, and the development of the field, and its results, suggest that we are in the presence of an authentic mathematical phenomenon. (If inner model theory keeps developing in a way that resembles current trends, and if indeed there are, say, incomparable theories, I expect our tools would reveal this by associating to the theories appropriate hod mice whose iteration strategies would end up being incomparable. On the other hand, we have theorems indicating that, so far, this does not appear to be possible; see for example Steel's preprint Normalizing iteration trees and comparing iteration strategies here.)
For more on large cardinals, why we "believe" in them, and references for much of the above, I recommend this MO thread.
Current research suggests a way in which this search for maximal interpretability power can actually reach as far as possible: This is the setting of Woodin's "Ultimate $\mathsf{L}$" theory, and its accompanying set of conjectures, notably, the $\Omega$-conjecture, that would allow us to prove that theories of large cardinals are indeed linearly ordered by interpretative power, and cofinal in the collection of theories, in terms of consistency strength. What is interesting about the modern approach is that we expect that there is not one "largest" correct theory, but rather, there may be many different theories that are mutually interpretable, and it makes no sense to prefer one to the other (on mathematical grounds), leading to a multiverse view of set theory.
That said, Woodin has suggested that, should Ultimate $\mathsf{L}$ exist and have the properties we expect, the axiom $\mathsf{V}=\mathrm{Ultimate}\mbox{-}\mathsf{L}$ plus large cardinals would provide us with a theory that in a sense is special, since it cannot be forced, and would allow us to interpret just about any other natural theory in inner models of forcing extensions of initial segments of $\mathsf{V}$. That the theory cannot be forced is partial evidence towards its "natural" rigidity, in the same sense that the theory of $\mathsf{L}$ is rigid: We have some examples of statements $\phi$ independent of $\mathsf{V}=\mathsf{L}$, but for all of them we can reasonably argue which of $\phi$ and $\lnot\phi$ is true in $\mathsf{L}$, say $\phi$, since any model of $\mathsf{V}=\mathsf{L}+\lnot\phi$ is, for instance, not well-founded.The same would be the case with $\mathsf{V}=\mathrm{Ultimate}\mbox{-}\mathsf{L}$, with the added advantage that it accomodates all large cardinals and has additional "correctness" properties. But this is all quite speculative at the moment.
A: (This answer was given to a previous version of the question.)
Your intuition is very wrong, actually.
Suppose that $\kappa$ is inaccessible. It's not hard to show that $V_\kappa$ is a model of $\sf ZFC$. However there is a closed and unbounded subset of $\kappa$, such that for every $\alpha$ in that subset, $V_\alpha$ is a model of $\sf ZFC$. It follows, if so, that even if $\kappa$ were the least inaccessible and we consider $V_\kappa$ as a model of $\sf ZFC$, it is true in that model that every set is an element of a transitive model.
Formally speaking, the statement $\varphi(M,E)=\ulcorner (M,E)\models\sf ZFC\urcorner$ can be formalized in $\sf ZFC$. This is an internal point of view, though. Because in $V$ we can write the recursive formula for the axioms of $\sf ZFC$ and so we can state that in the structure $(M,E)$ all the objects which $V$ thinks are axioms of $\sf ZFC$ are true in $(M,E)$.
$V$'s own metatheory may disagree about what are the axioms (e.g. if $V$ has non-standard integers), but as an internal statement it is possible to formalize that.
Now we just say the following: $$\forall x\exists M\exists E(x\in M\land\varphi(M,E))$$
You can require even more. You can require that $E=\in$, or just that $E$ is a well-founded relation. In fact you should require more, as the above is very naive and trivially satisfied the moment you have one model of $\sf ZFC$. You should require the following instead, $$\forall x\exists M\exists E(x\in M\land\varphi(M,E)\land\forall y(y\in x\leftrightarrow y\in M\land \ulcorner(M,E)\models y\in x\urcorner))$$
Now we require that $x$ is not only an element of the model, but in fact a subset and $M$ computes $x$ correctly. That assumption is probably closer to what you're looking for, and is in fact much stronger than the above one. But as I pointed out, still weaker than the existence of an inaccessible.
A: This topic is both mathematically and philosophically rich. What precisely do we mean when we say that a set-theoretic principle is "restrictive"? Can we give a formal account of this notion that aligns with our prereflective intuitions about restrictiveness? This is how I understand your question.
Penelope Maddy attempted to do exactly that in her book Naturalism in Mathematics, following up on her article, V=L and Maximize, in which she made a very specific technical proposal of what it is that makes a theory formally `restrictive'. The aim is to reify the informal concept of restrictiveness that is present in your question, and which was present in Maddy's earlier accounts in the links that Andres cites, so that we may treat restrictivenes as a mathematically formal notion.
Was the proposal successful? Well, you'll have to judge for yourself. The proposal has been criticized from a number of angles, including B. Loewe, A first glance at non-restrictiveness, and B. Loewe, A second glance at non-restrictiveness, where he proves among other things that ZFC itself is formally restrictive on Maddy's account. My article A multiverse perspective on the axiom of constructibility finds further serious problematic issues. In the end, it seems to me that one will need to revise the proposal, if a satisfactory formal account of restrictiveness is desired.
Meanwhile, in my paper I also address head-on the purely intuitive argument Maddy and others make against the axiom of constructibility — what I call the V≠L via maximize position — which rejects the axiom of constructibility V=L on the basis that it is restrictive. After first explaining various senses in which $V=L$ is compatible with set-theoretic strength (for example, every countable model of ZFC has an end-extension to a model of $V=L$), I argue that the $V\neq L$ via maximize position implicitly takes a stand in the pluralist debate in the philosophy of set theory by presuming an absolute background concept of ordinal. In particular, the $V\neq L$ via maximize argument appears to lose its force, in contrast, on an upwardly extensible concept of set, in light of the various facts showing that models of set theory generally have extensions to models of V=L inside larger set-theoretic universes. That is, with a robust multiverse concept of set, we no longer see $V=L$ as limiting, since larger universes can have large cardinals again, and still larger universes recover $V=L$ again. This perspective also avoids Steel's objections on the instrumentalist dodge, which are similarly based on an absolute background concept of ordinal.
You can read the paper, but here is the concluding paragraph:

Ultimately, the multiverse vision entails an upwardly extensible concept of set, where any current set-theoretic universe may be extended to a much larger, taller universe. The
current universe becomes a countable model inside a larger universe, which has still larger extensions, some with large cardinals, some without, some with the continuum hypothesis, some without, some with $V=L$ and some without, in a series of further extensions continuing longer than we can imagine. Models that seem to have $0^\sharp$ are extended to larger models where that version of $0^\sharp$ no longer works as $0^\sharp$, in light of the new ordinals. Any given set-theoretic situation is seen as fundamentally compatible with $V=L$, if one is willing to make the move to a better, taller universe. Every set, every universe of sets, becomes both countable and constructible, if we wait long enough. Thus, the constructible universe $L$ becomes a rewarder of the patient, revealing hidden constructibility structure for any given mathematical object or universe, if one should only extend the ordinals far enough beyond one's current set-theoretic universe. This perspective turns the $V\neq L$ via maximize argument on its head, for by maximizing the ordinals, we seem able to recover $V=L$ as often as we like, extending our current universe to larger and taller universes in diverse ways, attaining $V=L$ and destroying it in an on-again, off-again pattern, upward densely in the set-theoretic multiverse, as the ordinals build eternally upward, eventually exceeding any particular conception of them.

For your question, the point is that negated large cardinal axioms similarly needn't be viewed as limiting, if we think there is a much larger universe that has large cardinals, even if that universe is countable inside a model of $V=L$, in an endless tower of universes.
A: Why do you have such an intuition?
Anyway, the closest thing I can think of that may be relevant is:
If $ZFC + \textit{There exists an inaccessible cardinal}$ is consistent, then in any model $M$ of this theory, let $\kappa \in M$ and $M \models \kappa \textit{ is least inaccessible cardinal}$. Then 
$$M \models (V_{\kappa} \models ZFC + \textit{There are no inaccessible cardinal})$$
Hence in every such $M$, $V_\kappa$ is a set model of $ZFC + \text{There are no inaccessible cardinals}$. According to $M$, $V_\kappa$ is bounded (of cardinality $\kappa$). 
