Mike, I would not be surprised if at some point we bump into each other in the future or may have even done so in the past. The embbedded community is still farily small even with the explosive growth in the Android world.

That was one of the big pushes behind the debug core in the kernel was to allow other archs to fill out a "processor specific stub" + "a polled I/O driver", then you could get a whole debugger working.

@jwessel: Yeah, it's nice to have it in there if you've got a way to interact with the user like on x86. Tough to do that on a router or printer though. I'd love to see KGDB over USB or KGDBoE come back. The Ethernet option was particularly handy because you could have th console and debug at the same time without the interleaving.

You just have to know the little tricks to turn it on. Another really useful device to have (which time didn't permit for in Mike's presentation) is the EHCI debug adapter, if you are using x86 HW. If you don't have a serial and don't have a jtag, this device will get you early_printk() as well as kdb / kgdb at 1 mbit speeds.

@kernelkernel: That presentation can be found at both the Linux Foundation website (I did it at the Android bulder's summit earlier this year) and at the OpenOCD website. I didn't do the original recording. So, I'm afraid I couldn't control the quality :(.

@timd: if you look at the boot output on the serial console, you'll see a line of output that tells you how much memory the kernel is using. If you try that from user space, the memory size includes buffers and chaching space as well. So, you can't tell how much the kernel proper is using. An you should always beconcerned about memory usage on an embedded platform.

@Michael, thank you very much, I did not know the size of that kmalloc() pool is #defined as opposed to kernel config. I will continue looking. PS. A world reknowned manufacturer could have done a better job differentiating where to use GFP_ATOMIC :|

@Michael: re JTAG interface chipset; yes, of course the FTDI USB chipset. I believe I have seen this in the TinCan and Flyswatter tools on the web. I'm looking to piece together an Eclipse-based (nominally ARM) based development platform with JTAG debugging, out of my own pocket. I'm a old UNIX kernel person from my first job at Bell Labs in the late 70's, then transitioned to embedded stuff in the 80's and early 90's. Doing "system engineering" work now but striving to get back to my first love doing embedded and looking to dig into embedded Linux. Thanks for the great overview course.

@arkamax: you're really talking about the kernel's emergency page pool (GFP_ATOMIC allocates from there). There's a #define that proovides a guidline for the size of the emergency page pool. But, at the moment, I can't recall the define's location :(. However, it's really bad form to use GFP_ATOMIC allocations indescriminantly. I'd recommend a detailed analisys of the driver to find where they really need ATOMIC vs. the normal GFP_KERNEL allocations and fix the driver.

I have a small embedded system with an ARM processor. How can I tell how much RAM the kernel is using? Is there a way to look at this from user-space? Should I be concerned about my programs using to much RAM?

jaranda: You can debug from power on all the way through the kernel user space depending on the tools. But certainly you can debug u-boot with a JTAG. We often refer to working on the boot loader and prior to having a working kernel printk as "board bring up".

Thanks Mike. I'm not a linux developer or user yet, but the ideas for instrumenting the system are really useful for future projects. They may be based on some kind of OS or just a state machine. Software instrumenting using dynamically loaded modules or stubs on the system code sounds good to me.

@Michael - yes, I do - there is about 25 places where kmalloc() is used, and the way the source is built, it's hard to understand what's called from where. Is there any way to increase mem pool for kmalloc()? They recommend 4 Mbytes at least. Thanks.

@Michael: I didn't catch the whole statement, but I believe you mentioned something about a "chipset"(?) or a device make makes a decent JTAG interface, particularly ARM-based boards. Could you repeat what you were referencing. Thanks.

Speaking of kernel compiling... I've got a third party driver that is overly zealous on kmalloc() with GFP_ATOMIC flag set - it basically runs out of kmalloc()'able default memory pool I have on my ARM. Vendor says I can increase the size of kmalloc() memory pool - but I cannot find how to do that. Using LTIB from Freescale. Any comment helps. Thanks :)

i remember an old debugging vector scope in Byte. it had the address bus split on a pair of a/d converters. x-y scope display draws an signature of where the code is executing. a great tool to "sniff" the bus.

Mike, one thing that is not mentioned in the presentation is that KMS (Kernel Mode Setting) support was merged as well. The means you can take a kernel oops or panic while running on a graphics console and jump to the kdb prompt. (fully merged in 2010).

The mainline version of KDB connects to the debug core. The KGDB / gdbstub connects to that very same debug core. The configuration is all done through the kgdboc (KGDB Over Console) which is the I/O driver layer.

A few weeks ago, Ford Motor Co. quietly announced that it was rolling out a new wrinkle to the powerful safety feature called stability control, adding even more lifesaving potential to a technology that has already been very successful.

It won't be too much longer and hardware design, as we used to know it, will be remembered alongside the slide rule and the Karnaugh map. You will need to move beyond those familiar bits and bytes into the new world of software centric design.

People who want to take advantage of solar energy in their homes no longer need to install a bolt-on solar-panel system atop their houses -- they can integrate solar-energy-harvesting shingles directing into an existing or new roof instead.

Focus on Fundamentals consists of 45-minute on-line classes that cover a host of technologies. You learn without leaving the comfort of your desk. All classes are taught by subject-matter experts and all are archived. So if you can't attend live, attend at your convenience.