Hi @Pavol_Kianicka,
The only way I have used the sliding contacts is when the two surfaces are in contact with each other. I was able to create an assembly with the orange and tan parts (2 parts total) in your image above. I applied the sliding constraint and a load and I was able to get the problem to solve but the results looked incorrect. So my guess if the sliding contact cannot be applied in this way. Also with the tolerance off on the Sliding Contact this will be driving up your solution time a lot. this could explain why your run time is so long.
I would suggest modeling some sort of bushing to connect your parts together, keep it as simple as possible. In the image above you want something that will be touching those three surfaces. Unless your interested in the stress in the joints I would not model your actual hardware, this will just complicate the analysis. Do not model ball bearings! 
You can adjust the stiffness of the “bushing” part to better match the ball bearings buy changing the material properties. If you want to.
A few things to keep in mind. the sliding contact is a linear constraint so its only valid for “small” rotations. If our parts rotate too much then the results will be off.
The Sliding Contact constraint transfer the loads over the entire 360 degrees while in real life the loads are transferred across less than 180 degrees. If your looking at stresses around the connection points your results will be off. If this is what you are interested in then Physical Contacts may be needed.
I hope this helps a little. You are trying to tackle a very interesting problem.
Christopher