Author Topic: How does ST 32F4 know when an ISR has finished?  (Read 2272 times)

0 Members and 1 Guest are viewing this topic.

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3694
  • Country: gb
  • Doing electronics since the 1960s...
How does ST 32F4 know when an ISR has finished?
« on: May 09, 2022, 01:38:19 pm »
Referencing this
https://www.eevblog.com/forum/microcontrollers/how-are-interrupts-handlers-implemented/msg4164370/#msg4164370

I have googled around and there is a vast number of hits which chew all around this topic but I have not found any explanation of how one can have an ISR which calls other functions (subroutines) but still does not enable interrupts (of same or lower priority) until the very final RET.

I have written a number of ISRs for the 32F417 and all work, and some call other functions. But there is no easy way to check when other interrupts get re-enabled (one would need to waggle some GPIO pins from within various ISRs).

The 32F4 handles ISRs transparently, auto saving the whole context on the stack and restoring it afterwards. So in C an ISR is just a normal function (apart from having to clear the interrupt pending (IP) bit in many cases, in the peripheral). There is no IRET/RETI being generated, and there is no "interrupt" keyword used with the C ISR.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline bingo600

  • Super Contributor
  • ***
  • Posts: 1987
  • Country: dk
Re: How does ST 32F4 know when an ISR has finished?
« Reply #1 on: May 09, 2022, 02:29:53 pm »
Try to lookup
Cortex NVIC
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3694
  • Country: gb
  • Doing electronics since the 1960s...
Re: How does ST 32F4 know when an ISR has finished?
« Reply #2 on: May 09, 2022, 02:47:23 pm »
Yeah; I've spent many a happy hour decoding that stuff.

I was hoping somebody would come back with a straight answer :)
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 8167
  • Country: fi
Re: How does ST 32F4 know when an ISR has finished?
« Reply #3 on: May 09, 2022, 02:52:56 pm »
So you want to know the internal implementation details. You can't, it's all proprietary IPR. And why would you?

To me, it again seems you have some massive misunderstanding you are trying to work around.

"But there is no easy way to check when other interrupts get re-enabled". Wat? This makes no sense at all. Nothing is being disabled, and hence, nothing needs re-enabling.

It's all utterly trivial: higher priority interrupts interrupt the lower priority ones. Lower priority interrupts, if they happen during execution of high-priority ISR, are pending and automatically get executed after higher priority interrupts return. It doesn't matter how this is internally implemented, because it just works. You don't need to manage this by yourself at all.

If you want to dynamically demote the ISR priority, i.e., do something important quickly, then continue processing at lower priority, letting other mid-urgency ISRs inbetween, just trig a lower-priority software interrupt. This is what I do all the time:

Code: [Select]
NVIC_SetPriority(SPI_IRQn, 2); // quite high priority
NVIC_SetPriority(PROCESSING_IRQn, 10); // quite low priority
NVIC_SetPriority(OTHER_IRQn, 5);

void spi_handler()
{
   // read data, do some quick processing, check error conditions, whatever
   NVIC->STIR = PROCESSING_IRQn; // trigger processing_handler()
}

void processing_handler()
{
  // do whatever slow here, as long it finishes before next triggering
}

void other_handler()
{
  // spi_handler interrupts this; processing_handler does not
}

This also makes code portable and easy to understand, because the processing function is again just a regular function, so you can just... call it! Or let the NVIC "call" it.

There is also this "sub-priority" thing but IMHO, it's more like fine-tuning thing, I have never came up with any use for it in any actual project, so I just always
NVIC_SetPriorityGrouping(0);
so the priority is just one simple number; lower number simply always pre-empts higher number. KISS.

Internal implementation is surely quite complex compared to old simple CPUs, but that exactly makes using it easier.
« Last Edit: May 09, 2022, 02:58:25 pm by Siwastaja »
 

Offline hans

  • Super Contributor
  • ***
  • Posts: 1637
  • Country: nl
Re: How does ST 32F4 know when an ISR has finished?
« Reply #4 on: May 09, 2022, 03:04:41 pm »
When an interrupt occurs the handler is called 'like any other' function. It's jump address is looked up the interrupt vector table. I quoted 'like any other': because there is a special case for the LR register (link register) which contains the address back to the original program. Normally it would contain the instruction of the callee, but in an IRQ it contains a magic number like FFFF FFF9.

The CPU then *knows* when an interrupt is finished that this number is loaded back into PC, which normally would happen when a return is made from a function. E.g. by using BX LR (branch target address from register LR)  is executed, but also an instruction like POP PC, where the original PUSH was LR (the load/store multiple).

E.g.  take a look at this interrupt handler:

Code: [Select]
08000a64 <TIM7_IRQHandler>:
    void TIM7_IRQHandler() {
 8000a64: b508      push {r3, lr}
        StopwatchTimebase::tick(65536);
 8000a66: f44f 3080 mov.w r0, #65536 ; 0x10000
 8000a6a: 2100      movs r1, #0
 8000a6c: f7ff fc3e bl 80002ec <_ZN17StopwatchTimebase4tickEy>
        TIM7->SR = 0;
 8000a70: 4b01      ldr r3, [pc, #4] ; (8000a78 <TIM7_IRQHandler+0x14>)
 8000a72: 2200      movs r2, #0
 8000a74: 611a      str r2, [r3, #16]
    }
 8000a76: bd08      pop {r3, pc}
 8000a78: 40001400 .word 0x40001400

On entry you can see a push r3, lr to save these values to stack memory. On exit, the values are popped back from stack memory into the registers, however, this time the LR value is put into PC.

The PC sees it's loaded with the magic value and sets the processor up back in the original mode (e.g. if main or process stack was used, if another interrupt was handled etc.). You can look up these magic values and processor modes on page 596, section B1.5.8 "Exception return behaviour", in this PDF: https://documentation-service.arm.com/static/5f8fef3af86e16515cdbf816?token=

(ARMv7-m reference manual)

Obviously the interrupt status bit also needs to be reset, otherwise the interrupt vector will keep the CPU re-entering the handler over and over again.

I do wonder if the NVIC has a hardware table as well that notes which interrupt vectors have already been started execution (it's a nested controller, after all), and in which order the vectors need to be popped and/or started (depending also on priorities).
« Last Edit: May 09, 2022, 03:08:07 pm by hans »
 
The following users thanked this post: nctnico, zzattack

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3694
  • Country: gb
  • Doing electronics since the 1960s...
Re: How does ST 32F4 know when an ISR has finished?
« Reply #5 on: May 09, 2022, 03:18:13 pm »
Thanks; very interesting.

That comparator on the PC value is how they achieve an ISR with no "IRET/RETI" ending, while still being able to do what the hell it likes in the way of calling other functions and generally messing about on the stack.

In my project I have not needed to enable lower priority ints during any ISR, because all the ISRs have been very short to start with.

Very clever, especially the way one can let in lower priority interrupts before the ISR ends. Can't really see how that was implemented, but fair enough. I've done all that in the Z80 sphere.

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline emece67

  • Frequent Contributor
  • **
  • !
  • Posts: 614
  • Country: 00
Re: How does ST 32F4 know when an ISR has finished?
« Reply #6 on: May 09, 2022, 03:57:45 pm »
.
« Last Edit: August 19, 2022, 05:25:01 pm by emece67 »
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3694
  • Country: gb
  • Doing electronics since the 1960s...
Re: How does ST 32F4 know when an ISR has finished?
« Reply #7 on: May 09, 2022, 05:46:12 pm »
So you could fake an "RETI" by pushing that value on the stack and doing a return (plus some other stuff).

On the Z80-level stuff, I used to do

 some code
 some code
 ld hl, next
 push hl
 reti
next:
 some code  ; lower priority interrupts are now enabled
 some code
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14439
  • Country: fr
Re: How does ST 32F4 know when an ISR has finished?
« Reply #8 on: May 09, 2022, 06:16:31 pm »
An ISR has "finished" when it returns, but you got answers already.
I'm guessing there's something specific you want to achieve or work around?
 

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4199
  • Country: us
Re: How does ST 32F4 know when an ISR has finished?
« Reply #9 on: May 09, 2022, 07:09:58 pm »
Quote
Normally [the LR register] would contain the instruction of the callee, but in an IRQ it contains a magic number
I've wondered whether this "magic number" is actually just an address of some hardwired code off in "ROM" somewhere.  ("Microcode on a RISC?  Inconceivable!")  All it does is pop some extra context in addition  to the PC.  That would make it vaguely compatible with older ARM architectures, as well as (vaguely) many "external interrupt controllers" as found in more primitive systems.
Code: [Select]
OnInterrupt:
    save PC
    save interrupt level/etc
    save other context, including registers.
    branch to vectorTable[interruptNumber]

OnISRReturn:
    restore other context
    restore interrupt level
    restore PC
Historically, the function of an "Interrupt Controller" is to turn individual requests into a PC to call.  Back in the Z80 days, the EIC would actually grab the bus and jam in a "call xxx" instruction, where xxx was derived from the pins making the request.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3694
  • Country: gb
  • Doing electronics since the 1960s...
Re: How does ST 32F4 know when an ISR has finished?
« Reply #10 on: May 09, 2022, 08:01:58 pm »
Quote
I'm guessing there's something specific you want to achieve or work around?

I have no immediate application for this; just wondered how the arrangement worked.

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline bson

  • Supporter
  • ****
  • Posts: 2269
  • Country: us
Re: How does ST 32F4 know when an ISR has finished?
« Reply #11 on: May 09, 2022, 08:06:13 pm »
There's a bit in LR to indicate whether it's an exception return or a regular PC value.  When you do "b lr" with the bit set in LR the NVIC performs an exception unwind rather than just LR => PC.  You can also check it yourself to see if you got an exception in an exception.  And GDB uses it to unwind the stack correctly, showing you where in the backtrace you have exception boundaries.
 

Offline hans

  • Super Contributor
  • ***
  • Posts: 1637
  • Country: nl
Re: How does ST 32F4 know when an ISR has finished?
« Reply #12 on: May 09, 2022, 08:20:22 pm »
Quote
Normally [the LR register] would contain the instruction of the callee, but in an IRQ it contains a magic number
I've wondered whether this "magic number" is actually just an address of some hardwired code off in "ROM" somewhere.  ("Microcode on a RISC?  Inconceivable!")  All it does is pop some extra context in addition  to the PC.

Hmm good question.

The arch ref manual isn't very helpful about instructions concerning the coprocessor. I do see it's possible to load a bunch data into the coprocessors registers from a specific memory location. So there might be something to it: a small ROM of a few dozen bytes that contains some crt0.s (or are we at -1 ? :-/O) for IRQ handling. There also seems to be enough space there (8 bytes) that allows 2 ARM instructions to be executed: e.g. restore coCPU state (LDC) and then restore register state (pop or load-multiple, although that one can't modify PC). I'm also pretty sure that return to thread mode with main/process stack would then fall through each other, with the latter instruction being the final return to program.

After all, once you have built a neat machine that's capable of handling generic instructions to load/store memory and data, why would you bother hooking more logic in the critical path to support (hardcoded) IRQs? (only if you need/want the least amount of cycles for IRQ entry, I suppose)

But that's just my best guess.  For all we know, there may exist some undocumented instruction that does IRQ magic all in 4 bytes, which is particularly hard to poke at if you want to attempt to reverse engineer it (because who is exhaustively execute all undocumented opcodes at that particular system state -- well I suppose someone already did, but for me it's easier to be ignorant of any further details right now).
 

Offline langwadt

  • Super Contributor
  • ***
  • Posts: 4414
  • Country: dk
Re: How does ST 32F4 know when an ISR has finished?
« Reply #13 on: May 09, 2022, 09:01:17 pm »
Quote
Normally [the LR register] would contain the instruction of the callee, but in an IRQ it contains a magic number
I've wondered whether this "magic number" is actually just an address of some hardwired code off in "ROM" somewhere.  ("Microcode on a RISC?  Inconceivable!")  All it does is pop some extra context in addition  to the PC.  That would make it vaguely compatible with older ARM architectures, as well as (vaguely) many "external interrupt controllers" as found in more primitive systems.

I'd expect that it is just the NVIC handfeeding a hardwired sequence of instructions that are needed push/pop the right registers, to the core. It is a bit more clever that just push/call/pop, it skips the redundant pop/push when an interrupt follows an interrupt. 
It can do also do clever things like only push/popping FPU registers when and if they are used
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4028
  • Country: nz
Re: How does ST 32F4 know when an ISR has finished?
« Reply #14 on: May 10, 2022, 05:51:18 am »
Quote
Normally [the LR register] would contain the instruction of the callee, but in an IRQ it contains a magic number
I've wondered whether this "magic number" is actually just an address of some hardwired code off in "ROM" somewhere.  ("Microcode on a RISC?  Inconceivable!")  All it does is pop some extra context in addition  to the PC.  That would make it vaguely compatible with older ARM architectures, as well as (vaguely) many "external interrupt controllers" as found in more primitive systems.

I'd expect that it is just the NVIC handfeeding a hardwired sequence of instructions that are needed push/pop the right registers, to the core. It is a bit more clever that just push/call/pop, it skips the redundant pop/push when an interrupt follows an interrupt. 
It can do also do clever things like only push/popping FPU registers when and if they are used

One of the RISC-V spec documents has examples of normal code that you can put as your interrupt handler that implements functionality including interrupt chaining (not restoring then re-saving registers if there is an interrupt pending) and servicing a higher priority interrupt that comes in while a low priority interrupt is busy saving registers.

The execution times for each on a single-issue core are not dissimilar to Cortex-M Mo/M3/M4 "hardware" interrupt handling. It would be faster on dual-issue hardware.

Oh. I haven't seen C-M7 timings.
 

Offline Jeroen3

  • Super Contributor
  • ***
  • Posts: 4078
  • Country: nl
  • Embedded Engineer
    • jeroen3.nl
Re: How does ST 32F4 know when an ISR has finished?
« Reply #15 on: May 10, 2022, 06:15:59 am »
Try to lookup
Cortex NVIC
You need the ARM®v7-M Architecture Reference Manual, then the chapter about the exception model iirc.

Quote
An exception return occurs when the processor is in Handler mode and one of the following instructions loads a
value of 0xFXXXXXXX into the PC:
• POP/LDM that includes loading the PC.
• LDR with PC as a destination.
• BX with any register.

But there are loads of more conditions around this small snippet. I suggest you download the pdf.
https://documentation-service.arm.com/static/5f8fef3af86e16515cdbf816
« Last Edit: May 10, 2022, 06:17:51 am by Jeroen3 »
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4028
  • Country: nz
Re: How does ST 32F4 know when an ISR has finished?
« Reply #16 on: May 10, 2022, 06:51:39 am »
One of the RISC-V spec documents has examples of normal code that you can put as your interrupt handler that implements functionality including interrupt chaining (not restoring then re-saving registers if there is an interrupt pending) and servicing a higher priority interrupt that comes in while a low priority interrupt is busy saving registers.

The execution times for each on a single-issue core are not dissimilar to Cortex-M Mo/M3/M4 "hardware" interrupt handling. It would be faster on dual-issue hardware.

Oh. I haven't seen C-M7 timings.

I'm looking at an NXP document: https://www.nxp.com/docs/en/application-note/AN12078.pdf

It lists interrupt latency for various cores as:

CPU coreCycles
Cortex-M016
Cortex-M0+15
Cortex-M3/M412
Cortex-M710~12

This document shows toggling a GPIO pin on and off after a timer interrupt (which also sends a signal to an output pin) using the following code on an i.MX RT1050 (Cortex-M7) with zero wait state memory:

Code: [Select]
LDR.N R0, [PC, #0x78] ; GPIO2_DR
MOV.W R1, #8388608 ; 0x800000
STR R1, [R0]
MOVS R1, #0
STR R1, [R0]
BX LR ; not shown but I assume

With an oscilloscope they get figures of 10 cycles to enter the interrupt handler, 34 cycles to toggle the pin on, 32 cycles to toggle the pin off. (STR to IO space is much slower than the core speed)

They also look at latency of an interrupt from a GP)IO input pin (11 cycles) and latency to a blocked task in FreeRTOS 9.0.0 (490 cycles)
« Last Edit: May 10, 2022, 07:19:10 am by brucehoult »
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4028
  • Country: nz
Re: How does ST 32F4 know when an ISR has finished?
« Reply #17 on: May 10, 2022, 07:14:38 am »
E.g.  take a look at this interrupt handler:

Code: [Select]
08000a64 <TIM7_IRQHandler>:
    void TIM7_IRQHandler() {
 8000a64: b508      push {r3, lr}
        StopwatchTimebase::tick(65536);
 8000a66: f44f 3080 mov.w r0, #65536 ; 0x10000
 8000a6a: 2100      movs r1, #0
 8000a6c: f7ff fc3e bl 80002ec <_ZN17StopwatchTimebase4tickEy>
        TIM7->SR = 0;
 8000a70: 4b01      ldr r3, [pc, #4] ; (8000a78 <TIM7_IRQHandler+0x14>)
 8000a72: 2200      movs r2, #0
 8000a74: 611a      str r2, [r3, #16]
    }
 8000a76: bd08      pop {r3, pc}
 8000a78: 40001400 .word 0x40001400

On entry you can see a push r3, lr to save these values to stack memory. On exit, the values are popped back from stack memory into the registers, however, this time the LR value is put into PC.

Why is that pushing and popping r3? The hardware already does that. Just for stack alignment?
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3694
  • Country: gb
  • Doing electronics since the 1960s...
Re: How does ST 32F4 know when an ISR has finished?
« Reply #18 on: May 10, 2022, 08:39:35 am »
12 clocks at 168MHz is not bad.

The real shocker on the 32F4 is how slow any r/w to the peripherals is. See my previous threads on polled SPI, etc. Probably 10x slower than anyone used to "normal stuff" would have expected.

I guess there is a price to be paid for auto context saving. If doing it manually you can choose which registers to save.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4028
  • Country: nz
Re: How does ST 32F4 know when an ISR has finished?
« Reply #19 on: May 10, 2022, 09:03:26 am »
The real shocker on the 32F4 is how slow any r/w to the peripherals is. See my previous threads on polled SPI, etc. Probably 10x slower than anyone used to "normal stuff" would have expected.

I guess that's probably on everything. I tried toggling a GPIO pin on my RISC-V "HiFive1" board with the CPU running at 320 MHz. The same code writing to SRAM would run at 100 MHz (80 MHz with a NOP to make it symmetrical) but the best I could get to the GPIO pin was 17 MHz as measured on my oscilloscope.
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 3901
  • Country: gb
Re: How does ST 32F4 know when an ISR has finished?
« Reply #20 on: May 10, 2022, 10:20:16 am »
many more cycles during ISR on HC11 because it has to push and pop things on the stack, but I can still use it clocked @2Mhz as a type3 ps/2 keyboard controller.

Key-Debouncing is performed by hc11-hardware timer and one OC to assure the input is stable after x-msec, all in assembly to use less cycles and code possible.

It can done, and it works nicely up to three keys pressed at the same time (meta meta key, e.g. CTRL ALT DEL)! So, "cycles" is just a detail, success/failure only depends on what you want to achieve :D
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 3901
  • Country: gb
Re: How does ST 32F4 know when an ISR has finished?
« Reply #21 on: May 10, 2022, 10:46:50 am »
(
that means, for me, everything with a clock >=12Mhz is an Hulk-under steroids
compared to my little *grinder-MPU*  :o :o :o
)
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4028
  • Country: nz
Re: How does ST 32F4 know when an ISR has finished?
« Reply #22 on: May 10, 2022, 11:13:18 am »
many more cycles during ISR on HC11 because it has to push and pop things on the stack, but I can still use it clocked @2Mhz as a type3 ps/2 keyboard controller.

Key-Debouncing is performed by hc11-hardware timer and one OC to assure the input is stable after x-msec, all in assembly to use less cycles and code possible.

It can done, and it works nicely up to three keys pressed at the same time (meta meta key, e.g. CTRL ALT DEL)! So, "cycles" is just a detail, success/failure only depends on what you want to achieve :D

Sure, and a 1 MHz 6502 was plenty to keep up with human-scale things such as fast typing and key debouncing. Hell, Woz managed to get it processing 250,000 bps to/from the floppy disk, with I think just a 1 byte buffer, and bitstuff/decode that from/to 187,500 bps of user data (23.4 KB/sec) in real time.

But some things do need a bit more speed than that. For example driving WS2812 LEDs is *just* possible on an 8 MHz AVR, but not slower.
 

Offline wek

  • Frequent Contributor
  • **
  • Posts: 494
  • Country: sk
Re: How does ST 32F4 know when an ISR has finished?
« Reply #23 on: May 10, 2022, 11:53:13 am »
Why is that pushing and popping r3? The hardware already does that. Just for stack alignment?
Appears to be a historical leftover.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55757

Anybody can volunteer to improve on gcc... ;-)

JW
 
The following users thanked this post: hans, newbrain

Offline hans

  • Super Contributor
  • ***
  • Posts: 1637
  • Country: nl
Re: How does ST 32F4 know when an ISR has finished?
« Reply #24 on: May 10, 2022, 11:57:56 am »
@BruceHoult This function is not marked as "interrupt" in C(++). So it doesn't know that r3 is already pushed by hardware before IRQ handler entry.

When I add the interrupt attribute, I seem to get extra instructions that modify/align sp and push/pop r0 as a temporary variable in doing so:
https://godbolt.org/z/1rTzx76nd

You can uncomment the function prototype to see the results. In practice, if stack alignment is unnecessary, GCC seems to generate faster code without the interrupt argument. Seems to be related to this ticket:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55757

In which stack alignment for M devices, AFAIK doesn't really matter. Sad to see that this issue only is still open for 9,5 years.. I also can't find any flags to tell the compiler the code should be compiled for a Cortex-M3 r2p0 with some extra architecture fine tuning.
Interestingly it does the same on Cortex-m7 as well, which is ARMv8.
If the tick function is inlined (which can be a few dozen lines of ASM), then I see an uneven number of pushes on IRQ handler entry: e.g. r4, r5 and lr, and a sp move.

This has been an issue with GCC for me in general. I've experimented with MIPS GCC lately for Microchip devices, and in particular the code for interrupts seem to be extremely bloated in which the CPU first pushes literally a dozen registers (!) before even starting any work on the actual body of my interrupt function. Even when I enable the shadow register set it doesn't get better than half a dozen registers.. and sometimes pushing the same registers at callee and called function. ARM GCC seems to do a bit of a better job though.

Edit: ha coincidence @wek you found the same ticket :) You're right, anyone can improve the compiler project if needed..
« Last Edit: May 10, 2022, 12:18:17 pm by hans »
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf