Author Topic: techniques for writing non blocking code  (Read 17602 times)

0 Members and 1 Guest are viewing this topic.

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 1898
  • Country: ca
Re: techniques for writing non blocking code
« Reply #75 on: August 04, 2017, 03:46:22 pm »
I did note write that.

Of course you didn't. I fixed the quotation.

You appear not to understand the relevance of the classic "dining philosophers" problem to your position.

The deadlock problem has nothing to do with real-time performance. Avoiding deadlocks is algorithmic question. Think about it. It is possible to find a solution  for the "philosophers" problem before you even think of the hardware platform you're going to use.

We're discussing here real-time systems, which heavily depend on the hardware architecture.

Real-time performance is the ability to act at a specified point of time. When something happens, your program needs to take control and do something. The time between the triggering event and the moment where the program acts is called latency. Sometimes, if your event is predictable (such as periodic timer) you can master zero-latency. Other times, the event is unpredictable and thus zero-latency is impossible. The lowest latency you can master is most likely interrupt latency, or if you can dedicate your entire CPU to this single event, it is the duration of the shortest loop capable of detecting an event.

Now imagine you have two real-time tasks. Say, task A is doing something at fixed period with zero-latency. At the same time, task B is doing something when a rising edge from an external line arrives. Thus task B has interrupt latency. What happens if the events happen as follows:

- a rising edge for task B arrives
- the interrupt latency time passes so that task B has to run now
- at exactly the same time the period for task A expires

If you have only one CPU, it is absolutely impossible for these two tasks to run at the same time. What you can do about this? You can run task A. This increases the latency of task B, and it is no longer equal to interrupt latency, but is equal  to (interrupt latency + time to do the minimum processing for task A). Or, you can run task B, then the latency of task A will not be zero any longer, but will be equal to the time necessary to do some minimum processing for task B. If neither of these meets your timing requirements, nothing you can do. This is interference caused by multitasking. A can be served alone, B can be served alone, but A and B together cannot.

The only solution is to run tasks A and B on two different physical devices. Say, you get two PIC16s, one is doing task A, the other is doing task B. Then they somehow communicate when neither task A or task B is running. For 2 tasks we need 2 CPUs. For 8 tasks we need 8 CPUs.

What XMOS does? If I understand correctly, it has many CPU (cores) which can be dynamically assigned to the tasks as necessary. How many cores do we need to make sure 2 tasks never interfere with each other in a way I explained before. Two. How many cores do we need for 8 tasks? Eight. Well, we could probably get away with 6, but if there's any probability that all 8 events happen all at the same time, then 6 wouldn't be robust, would it? In short, to handle N tasks we need N CPUs. This is the same number of CPUs as if we had one CPU dedicated to one task. This is  practically the same as a system with N different MCUs on the board.

What most others do? They have peripheral modules which can handle common tasks, so that the time-critical portions of the tasks can be performed entirely by the peripheral modules without any CPU intervention. Say, you can have a PIC with 16 input capture modules. Each of the modules can capture edges with 10-20 ns resolution. Or you can have a number of PWM modules which will produce accurate signal with transitions timed at 10 ns resolution. And so on, and so on. As a result, a single PIC can perform way more tasks than a multi-core solution, and they totally do not interfere with each other in a sense that the time-critical actions are never postponed.

The question is, what to do if the real time requirements are so complex that they cannot be handled by peripherals? Very simple, put an extra MCU or 2 on the board for just that purpose. Not enough? Go to FPGA.
 

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 10690
  • Country: gb
    • Having fun doing more, with less
Re: techniques for writing non blocking code
« Reply #76 on: August 06, 2017, 10:11:08 am »
You appear not to understand the relevance of the classic "dining philosophers" problem to your position.

The deadlock problem has nothing to do with real-time performance. Avoiding deadlocks is algorithmic question. Think about it. It is possible to find a solution  for the "philosophers" problem before you even think of the hardware platform you're going to use.

We're discussing here real-time systems, which heavily depend on the hardware architecture.

If you think about it, those two paragraphs are mutually contradictory. The issue is that real-time systems' required behaviour doesn't depend heavily on hardware architecture. (But a choice of inadequate hardware and software can prevent it).

Remember, realtime != fast.


Quote
If you have only one CPU, it is absolutely impossible for these two tasks to run at the same time. What you can do about this? You can run task A. This increases the latency of task B, and it is no longer equal to interrupt latency, but is equal  to (interrupt latency + time to do the minimum processing for task A). Or, you can run task B, then the latency of task A will not be zero any longer, but will be equal to the time necessary to do some minimum processing for task B. If neither of these meets your timing requirements, nothing you can do. This is interference caused by multitasking. A can be served alone, B can be served alone, but A and B together cannot.

Not necessarily: providing constraints are met, interference can be absent. Meeting constraints can be aided/prevented by appropriate hardware+software mechanisms.

Quote
What XMOS does? If I understand correctly, it has many CPU (cores) which can be dynamically assigned to the tasks as necessary.

The tasks are statically allocated; that's required for the IDE to define execution times without executing code and measuring a subset of the possible times.

The interesting point about the XMOS devices is that the combination of hardware+software enables such predictions in a way that is impossible for bog-standard processors.

Quote
The question is, what to do if the real time requirements are so complex that they cannot be handled by peripherals? Very simple, put an extra MCU or 2 on the board for just that purpose. Not enough? Go to FPGA.

Extra MCUs won't, on their own, solve fundamental real-time problems.

The XMOS devices are no more magic than FPGAs and bog-standard processors. But the XMOS devices do have a set of unique beneficial characteristics - which illuminate the limitations of mainstream technologies.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 1898
  • Country: ca
Re: techniques for writing non blocking code
« Reply #77 on: August 06, 2017, 02:29:24 pm »
The issue is that real-time systems' required behaviour doesn't depend heavily on hardware architecture.

What do you mean by that? Are you suggesting that any hardware has equal (or similar) real-time capabilities?

Remember, realtime != fast.

Depends on what you mean by "fast". Not fast as a runner, but fast as a gun fighter.

 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 10471
  • Country: us
  • DavidH
Re: techniques for writing non blocking code
« Reply #78 on: August 06, 2017, 02:50:51 pm »
The issue is that real-time systems' required behavior doesn't depend heavily on hardware architecture.

What do you mean by that? Are you suggesting that any hardware has equal (or similar) real-time capabilities?

It means that how the software is designed has more of an effect on real time performance than the hardware.  Bad software can ruin the real time performance of any hardware.

Quote
Remember, realtime != fast.

Depends on what you mean by "fast". Not fast as a runner, but fast as a gun fighter.

Real time means bounded latency.  Almost all systems using high performance multi-core processors have an order of magnitude worse latency than 20+ year old systems using processors with 4 orders of magnitude less performance because of software limitations.

For example my HP50g calculator with a 50MHz 32 bit processor has higher latency than the HP48g it replaced which had a 4MHz 4-bit processor.  From the perspective of the user, it is *slower* and less power efficient.  That is the cost of progress.

 

Offline legacy

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: techniques for writing non blocking code
« Reply #79 on: August 06, 2017, 04:59:10 pm »
Because pipeline
worst caste: long pipeline with short icache.
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 1898
  • Country: ca
Re: techniques for writing non blocking code
« Reply #80 on: August 06, 2017, 05:46:49 pm »
It means that how the software is designed has more of an effect on real time performance than the hardware.  Bad software can ruin the real time performance of any hardware.

Hardware sets the limit which you cannot jump over. Then it is up to you to meet this limit with your software or overbloat your software beyond any reason. If most people chose the overbloat, does it really mean that real time performance doesn't depend on hardware?

Remember, realtime != fast.

Depends on what you mean by "fast". Not fast as a runner, but fast as a gun fighter.

Real time means bounded latency.  Almost all systems using high performance multi-core processors have an order of magnitude worse latency than 20+ year old systems using processors with 4 orders of magnitude less performance because of software limitations.

These are not software limitations. This is just the way the software is written today.

However, you always can use assembler to re-write your time-critical sections. This way you remove any dependency on software, and you will get the best possible latency from the device. But then you will find out that the so called "performance" of modern processors is predicated by pipelines and caches which increase throughput, but do not help with latency (especially worst-case latency). So, the super-duper ARM processor may have worse latency than a small PIC.

 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 18543
  • Country: nl
    • NCT Developments
Re: techniques for writing non blocking code
« Reply #81 on: August 06, 2017, 07:01:08 pm »
Perhaps but better pheripherals may have more relaxed interrupt reaction times. Also on a small 8 bit micro you may need to save the registers on the stack first which is something a Cortex Mx does in hardware.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 10471
  • Country: us
  • DavidH
Re: techniques for writing non blocking code
« Reply #82 on: August 06, 2017, 07:33:07 pm »
These are not software limitations. This is just the way the software is written today.

If it is how software is written today, then it is a software limitation.

This is like how Intel argued that their Itanium processor had a massive performance edge over other processors while showing off a few hundred lines of hand optimized assembly code.  This performance was not achievable with the compiler so where was the fault?  The hardware or compiler?  It is irrelevant and Intel's benchmark was irrelevant also.  The processor sucked except for killing the competition.

Quote
However, you always can use assembler to re-write your time-critical sections. This way you remove any dependency on software, and you will get the best possible latency from the device. But then you will find out that the so called "performance" of modern processors is predicated by pipelines and caches which increase throughput, but do not help with latency (especially worst-case latency). So, the super-duper ARM processor may have worse latency than a small PIC.

This is way beyond the limits imposed by pipelining and caches and language choice is irrelevant when the programmer is using things like just in time compiling, dynamic memory allocation, (1) and screwed up library code.

Garbage collection is usually always a killer.  (2) I remember Sun saying that JAVA would be used everywhere from the smallest microcontrollers to the largest multiprocessors.  I just laughed.

If the programming ecosystem does not allow programmers to achieve the performance that the hardware is capable of, then that performance does not exist.

(1) Dynamic memory allocation is feasible in real time environments but you have to use the right memory allocator and use it correctly.  I always thought Knuth's memory allocator could do it but I never tried implementing it.

(2) There might be a real time memory allocator which uses garbage collection but I have never seen it and the attempts to make one have been disappointing.  It seems like something which could be integrated into a RTOS but I assume this is one of those intractable hard problems.
 

Offline Bruce Abbott

  • Frequent Contributor
  • **
  • Posts: 616
  • Country: nz
    • Bruce Abbott's R/C Models and Electronics
Re: techniques for writing non blocking code
« Reply #83 on: August 06, 2017, 07:35:30 pm »
on a small 8 bit micro you may need to save the registers on the stack first which is something a Cortex Mx does in hardware.
Really?

https://www.adamheinrich.com/blog/2016/07/context-switch-on-the-arm-cortex-m0/
Quote
Context Switch

The context switch happens in an interrupt handler. Once an interrupt occurs, the NVIC hardware automatically stacks an exception frame (registers xPSR, PC, LR, r12 and r3-r0) onto the Process Stack (PSP) and branches to the interrupt handler routine in Handler Mode (which uses the Main Stack).

The context switch routine has to:

    Manually stack remaining registers r4-r11 on the Process Stack
    Save current task’s PSP to memory
    Load next task’s stack pointer and assign it to PSP
    Manually unstack registers r4-r11
    Call bx 0xfffffffD which makes the processor switch to Unprivileged Handler Mode, unstack next task’s exception frame and continue on its PC.
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 1898
  • Country: ca
Re: techniques for writing non blocking code
« Reply #84 on: August 06, 2017, 07:50:51 pm »
If the programming ecosystem does not allow programmers to achieve the performance that the hardware is capable of, then that performance does not exist.

What the hell is "programming ecosystem"? You're free to write your software as you please. There are lots of people out there who write good software.
 
The following users thanked this post: jpanhalt

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 10471
  • Country: us
  • DavidH
Re: techniques for writing non blocking code
« Reply #85 on: August 06, 2017, 08:02:40 pm »
If the programming ecosystem does not allow programmers to achieve the performance that the hardware is capable of, then that performance does not exist.

What the hell is "programming ecosystem"? You're free to write your software as you please. There are lots of people out there who write good software.

It is the tools and libraries available to the programmer for whatever reason.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 18543
  • Country: nl
    • NCT Developments
Re: techniques for writing non blocking code
« Reply #86 on: August 06, 2017, 08:07:27 pm »
on a small 8 bit micro you may need to save the registers on the stack first which is something a Cortex Mx does in hardware.
Really?

https://www.adamheinrich.com/blog/2016/07/context-switch-on-the-arm-cortex-m0/
There must be something wrong with that because it implies you can't use plain C functions as interrupt handlers which a Cortex Mx can use. OTOH it may be not all registers are used. Either way it doesn't negate the fact Cortex Mx controllers do in hardware what is typically done in software on an 8 bit microcontroller.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 10690
  • Country: gb
    • Having fun doing more, with less
Re: techniques for writing non blocking code
« Reply #87 on: August 06, 2017, 08:29:12 pm »
Garbage collection is usually always a killer.
...
(1) Dynamic memory allocation is feasible in real time environments but you have to use the right memory allocator and use it correctly.  I always thought Knuth's memory allocator could do it but I never tried implementing it.

(2) There might be a real time memory allocator which uses garbage collection but I have never seen it and the attempts to make one have been disappointing.  It seems like something which could be integrated into a RTOS but I assume this is one of those intractable hard problems.

While I agree with your other points, those points turn out not to be the case.

Many misconceptions about GCs are based on partial information of obsolete GC technology. Modern GCs have improved enormously, to the extent there are many commercially and technically successful real time systems written in Java; I have personally created a few.

Classic examples are "high frequency trading", which is critically latency-sensitive, and telecoms systems where soft real time performance is specified and achieved.

Of course, if hard realtime performance is specified, GC is almost always contra-indicated.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 1898
  • Country: ca
Re: techniques for writing non blocking code
« Reply #88 on: August 06, 2017, 08:31:11 pm »
It is the tools and libraries available to the programmer for whatever reason.

IMHO, the available tools let you fully realize the hardware potential if you're inclined to do so.
 

Offline legacy

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: techniques for writing non blocking code
« Reply #89 on: August 06, 2017, 08:34:10 pm »
What the hell is "programming ecosystem"? You're free to write your software as you please.

LOL  :-DD :-DD :-DD :-DD :-DD

 

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 10690
  • Country: gb
    • Having fun doing more, with less
Re: techniques for writing non blocking code
« Reply #90 on: August 06, 2017, 08:37:56 pm »
It is the tools and libraries available to the programmer for whatever reason.

IMHO, the available tools let you fully realize the hardware potential if you're inclined to do so.

That's a meaningless statement. Depending on interpretation, it is either wrong or trivially true.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline legacy

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: techniques for writing non blocking code
« Reply #91 on: August 06, 2017, 08:54:13 pm »
about transputer, this seems to be an interesting modern reboot (for fpga).
 

Offline andyturk

  • Frequent Contributor
  • **
  • Posts: 892
  • Country: us
Re: techniques for writing non blocking code
« Reply #92 on: August 06, 2017, 09:15:55 pm »
There must be something wrong with that because it implies you can't use plain C functions as interrupt handlers which a Cortex Mx can use. OTOH it may be not all registers are used. Either way it doesn't negate the fact Cortex Mx controllers do in hardware what is typically done in software on an 8 bit microcontroller.
Both the link and your interpretation are correct. An RTOS context switch has to save the entire state of the machine, so it can restore it later. A "normal" interrupt can just be straight C code.

Context switches with hard floating point can get pretty expensive because the floating point registers have to be saved/restored too.
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 1898
  • Country: ca
Re: techniques for writing non blocking code
« Reply #93 on: August 06, 2017, 09:31:42 pm »
Both the link and your interpretation are correct. An RTOS context switch has to save the entire state of the machine, so it can restore it later. A "normal" interrupt can just be straight C code.

Many modern CPUs have shadow register sets which are automatically swapped by hardware before launching the ISR, so you don't need to save registers. PIC16/18 too.
 

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 10690
  • Country: gb
    • Having fun doing more, with less
Re: techniques for writing non blocking code
« Reply #94 on: August 06, 2017, 09:32:58 pm »
about transputer, this seems to be an interesting modern reboot (for fpga).

Hmmm....

Personally I'm more interested in, for lack of a better term, the philosophy rather than a specific implementation of the philosophy.

Hence my being "satisfied" by xCORE+xC.

... so I suppose I'll regard that proposal as being interesting historically, but I'll reserve judgement about its practical interest.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline JPortici

  • Super Contributor
  • ***
  • Posts: 2567
  • Country: it
Re: techniques for writing non blocking code
« Reply #95 on: August 06, 2017, 09:33:58 pm »
What the hell is "programming ecosystem"? You're free to write your software as you please.
LOL

please guys, stop be so condescending. you are just as annoying as PSOC pretyped walls of text or even ADA and SUBLEQ crusaders, just to mention a couple.

that is the only reason why i bothered to comment before.

there is a chance that there are people who can think but are not as knowledgeable as you are or hasn't worked in avionics or don't know what an ingenious things xmos is yed and may be just humble programmers who think they can still write a good piece of software with their unsafe tools, as it happens and is still happening and will always happen as long as C exists... so a very long time.
and they may be asking what are, from their point of view, legitimate questions.

i see no reason to mock them (us)
this is not /g/ or reddit or whatever garbage community of elites
« Last Edit: August 06, 2017, 09:36:20 pm by JPortici »
 

Offline Bruce Abbott

  • Frequent Contributor
  • **
  • Posts: 616
  • Country: nz
    • Bruce Abbott's R/C Models and Electronics
Re: techniques for writing non blocking code
« Reply #96 on: August 06, 2017, 09:45:14 pm »
it may be not all registers are used.
Sure, but that's only useful if you limit your code to using a reduced register set. Even lowly 8 bit PICs and AVRs can do that.

Quote
Either way it doesn't negate the fact Cortex Mx controllers do in hardware what is typically done in software on an 8 bit microcontroller.
My definition of 'in hardware' would be using eg. shadow registers or bank switching, not just automatically saving registers on the stack (the STM8, 68HC11, and even the ancient MC6801 do that).
   
 

Offline KL27x

  • Super Contributor
  • ***
  • Posts: 3807
  • Country: us
Re: techniques for writing non blocking code
« Reply #97 on: August 06, 2017, 11:18:38 pm »
^ This what I was thinking. Who cares if it automatically saves registers? Newer 8 bit PIC have automatic context saving. Older ones don't. Only difference to me is 6 lines of assembly, 3 in the beginning of ISR and 3 at the end. This is pretty much saying "new model has 6 extra words of instructions and 3 extra bytes of memory compared to older model." You can also conisder those resources are automaticallly reserved for isr, which deprives the user in case he didnt need them for ISR. This is nice to the programmer but not an actual improvement in specs.

If it did this in parallel to core processor which reduces latency, then that's different.

I went my round with TGZZZZ several months ago. It's matter of perspective, which I accept as assembly programmer of simple device, I have complete control over actual hardware limitations. So I see North Guy's perspective as valid. But if you are using RTOS, you are using more complex toolchain which you can't completely rewrite or circumvent with inline assembly unless you are in a very specialized business. Most people are paid to get a job done, efficiently. Not to fine tune tools to make tools specific to a given end product.
« Last Edit: August 06, 2017, 11:39:12 pm by KL27x »
 

Offline Mechatrommer

  • Super Contributor
  • ***
  • Posts: 9385
  • Country: my
  • reassessing directives...
Re: techniques for writing non blocking code
« Reply #98 on: August 06, 2017, 11:44:56 pm »
please guys, stop be so condescending. you are just as annoying as PSOC pretyped walls of text or even ADA and SUBLEQ crusaders, just to mention a couple.
that is the only reason why i bothered to comment before.
there is a chance that there are people who can think but are not as knowledgeable as you are or hasn't worked in avionics
+1. let them try to define or impose on our reality with their own reality, its quite amusing watching from here. for us, their theories seems or sounds plausible yet confusing in multitude grade of levels, but above all, unproven or in another simpler word... nonsense... from a simple switch non blocking single cored code to multicores, parallelism, resources sharing, priority inversion, real time, ecosystem and whatever sheets thats going to be arised next... afaik most of those terms are very well defined and known since before our birth or probably since the invention of Babbage machine, then here they are come up with probably their own delusional meaning in this ever growing thread, just like the fate of the other threads.. the first rule of thumb is, when the slightest correlation between a noob thread and programming, you'll see long winded people with long winded theory try promoting this and that and how unholy C/C++ is... dont you ever try to correct the path to the laymen ship otherwise you'll be dragged away with all these confusing theories and will be doomed to look just like a fool... well... weekend is over... the fuck with the theories, we only care about practical and workable codes. we harness machines to do our jobs, not the other way around...
if something can select, how cant it be intelligent? if something is intelligent, how cant it exist?
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 1898
  • Country: ca
Re: techniques for writing non blocking code
« Reply #99 on: August 07, 2017, 12:21:09 am »
^ This what I was thinking. Who cares if it automatically saves registers? Newer 8 bit PIC have automatic context saving. Older ones don't. Only difference to me is 6 lines of assembly, 3 in the beginning of ISR and 3 at the end. This is pretty much saying "new model has 6 extra words of instructions and 3 extra bytes of memory compared to older model." You can also conisder those resources are automaticallly reserved for isr, which deprives the user in case he didnt need them for ISR. This is nice to the programmer but not an actual improvement in specs.

If it did this in parallel to core processor which reduces latency, then that's different.

PICs don't use extra cycles to store registers. They're stored automatically and restored back automatically. This is all done in hardware at the very moment the interrupt is taken. No single cycle is added.

PIC16F1* stores 8 different registers, which saves 16 instructions at the beginning of the ISR, which gives you 2 cycle latency (250 ns, or 166 ns on PIC16F14* which can use 48MHz clock). If you would do savings manually, it would give you 18 cycle latency - 4.5 us (plus 16 cycles when you leave the interrupt). 4.5 us is way bigger latency than 250 ns, so the auto-save is very important - it lets you achieve much higher performance.

The newest PIC18F - K42 series save 13 registers thus gaining 26 cycle time. In non-vectored mode the latency is 2 cycles too - you get control with all registers saved 125 ns after the interrupt. This is compared to 3.5 us if you had to save them manually. This is an extremely useful feature.

With PIC24EP/dsPIC33EP, some of which have so called "Alternate register sets", it avoids saving 15 registers, saving 15 cycles. But here the interrupt latency is 11 cycles, so you only get 157 ns at 140 MHz clock, but it's still better than 371 ns you get on the PIC24EPs without the shadow registers.
 
The following users thanked this post: KL27x


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf