There is a great deal that one can do without politics so long as one is willing to engage in building communities and institutions. If a group of people are concerned about the same issues and communicate those concerns with each other, or share general common interests and perspectives, it is fairly easy and straightforward to develop social ties where useful and enjoyable activities can be present. This is not to say that politics are lacking entirely from such institutions, but politics is one of those things that work best when it is nearly invisible, and where questions of power are secondary or even further down the level of importance to what is being done via a shared commitment.
Indeed, one can say that when the question of power and influence becomes primary is when things cannot help but go wrong. The reasons for this are not hard to understand. Many things in life are not a zero sum game. If I am spending time with friends and I enjoy what is going on, my enjoyment does not need to reduce the enjoyment of anyone else, and indeed, part of our enjoyment can spring from the pleasure that good times and good company bring to others as well as ourselves. Similarly, in economics, useful and beneficial exchanges are win-win because they provide more for us together than we could hope to gain working on our own. This is not the case with politics, which is very clearly a win-lose sort of situation. For someone to win an office, someone else (and often many people) must lose it. Politics, like sports, is a consummate competitive endeavor that provides a score that is public and where the importance of winning is such that people are not always honorable in their pursuit of the power and glory that comes from victory.
It is not only democratic politics that has this sort of problem, either, lest we think that democracies and republics failed in some way that aristocracies and more centralized regimes succeeded at. If one is engaged in a system of rule where one person is in charge, the identity of that person (to say nothing about their character) and those who control access to that person matters a great deal and competition is intense for such positions. Similarly, in aristocratic regimes, competition can be fierce for those offices that express the common will and that provide influence in shaping the direction of public policy. This conflict can be dealt with via power sharing, but the desire for offices and for power to enforce one’s own ideas and vision on a larger community always carries with it some sort of danger for the well-being of the institution or community as a whole. The building up and maintenance of a consensus need not make anyone feel like a loser, but trying to push through an idea that does not have common acceptance by using institutional power carries with it the price of a loss of faith in those institutions if they become too heavily controlled by people who view power as a means to an end that is not shared by the rest of the community involved.
The fact that we have so many difficulties with politics and its abuse does not mean that we have to hate it or fail to recognize why it exists. For societies and institutions to be run decently and in order, there has to be some sort of order and system involved in how they operate, and this will by necessity require some sort of authority. The existence of authority is one of those facts like gravity which will inevitably lead to politics, because how that authority is to be exercised, who has authority and on what grounds one gains or maintains authority are all inherently political questions. Like happiness is best sought indirectly through the operation of good deeds, though, politics is best seen as a byproduct of the operation of institutions and the beings who are within them. When we seek directly for power, or have to deal with those whose use of power suggests their desire for control and domination over others rather than their desire to act for the well-being of the community as a whole, what we are dealing with is abuse. As painful as that abuse is, the power that is abused can be and should be used in better ways, and one cannot eliminate that power because some sort of power will always be needed to do anything at all. And any power that exists can be abused by those who turn it to evil ends.