There are at least three kind of arbitrary relations in the mind sciences:
Between concepts/language and the world
Between the mind and the body
Between form and substance (which might include the above)
For example, we feel that there is no relation between the concept CUP and cups in the world. The concept CUP has neither shape nor size nor mass, while real world cups do. Similarly, concepts interact with each other logically — we can say “can you give me either the red or the blue cup?” while objects only interact with each other causally. Cups that fall on a hard floor break while the concept CUP does not break down when you say FLOOR.
The same goes for temporal arbitrariness. Consider the statement “Socrates died in 399 B.C.E.” Having once existed and died, Socrates is long gone but the statement regarding his death will now be true, independent of the rest of the history of the universe. Even if human beings become extinct as a species, Socrates would still have died in 399 B.C.E and the statement regarding his death would still be true. For these reasons, it seems possible to isolate an entity called a proposition that lives outside space and time and comes into relation (or is perhaps even contained within) with the human mind. To the extent that the human mind is a container for these kinds of entities, it is also primarily an abstract entity, whose foundational rules are abstract.
Consider the statement “dogs are animals”. The truth of this statement seems to have nothing to do with the actual character of dogs. You might have never seen one. Indeed, the statement would equally well apply to “grifmors are ringbats” as long as grifmors were known to be ringbats. The point is this: Conceptual structures are connected to the rest of the world, but only at the boundary. As long as the boundary conditions are known to be valid (Socrates dying in 399, dogs being animals etc) the rest of the conceptual structure is insulated from the universe. It is this encapsulation that leads to claims about modularity etc. We can see this boundary + interior reasoning explicitly in the minimalist program.
For similar reasons, we also feel that concepts are not like brain or body structures. Neurons interact using electrical impulses, while concepts do not. In fact, since concepts do not have any extension, they do not have any physical substance at all. What are they made of? According to Plato and Descartes, the essence of concepts is not a physical substance but a soul like substance, whose essence is reason. There is then an arbitrary relation between the body-like and soul-like substances, as well as their properties and states (the debate about the precise mental character of concepts, say, whether concepts are mental states or mental properties is ignored for now).
Both Gibsonians and Embodied Cognitivists have tried to dislodge this deep dualism, which comes from observations about the arbitrariness of the concept-body-world relation. I think they underestimate the strength of this position and therefore do not do enough to refute it thoroughly. For example, consider the CONTAINER schema often used by Lakoff and other cognitive linguists as an example of an imagistic element in human cognition. We could ask three questions of Lakoff about the nature of these schema:
Isn’t an image schema already an abstraction? Our experience of the world conjoins the wind blowing in our eyes, the smell of the jasmine flower and the green of the leaves. Where in all of this is container-hood? It seems as if our pre-conceptual experience actually does not have such a thing as containers.
Suppose, somehow we do experience rooms as things that make us act in certain ways (for example, making sure that we move towards the door when we want to leave the room, since the walls are inpenetrable. Even then the experience of not-being-able-to-leave-the-room is not the same as the abstract relation A-contained-in-B. Where does the latter come from?
In any case, the feeling of not-being-able-to-leave-the-room is a conceptual judgment. What is pre-conceptual about it? If anything it shows that bodily perception/experience is infused with conception rather than being the basis of post-perceptual conceptualization.
I agree with the embodied cognitivists that we shouldnt separate mind from the body; but in actually ‘fleshing’ that out, they are themselves as guilty of making the same mistakes (for example about preconceptual experience) as their modular opponents. A cognitive science that is truly non-arbitrary in its leanings will no more be body centric as it is form centric.