Hi,
> We've notice (as gray-haired BSD folk at heart) that Linux sometimes
> is very PC-centric. But if you want Linux to be portable, you'd
> better work towards losing these two assumptions; and not just in
> 'ifdefs' either. Even Intel might ship a Merced one day.
You're right, Linux is still pretty PC-centric. But worse is that the
people working on Linux are thinking PC centric. I'm a pessimist in that
respect; that's why I'm trying to fix things invissible for device drivers.
Being PC centric has so far not been a problem because m68k and Sparc where
these problems do exist also use their own drivers and have their own,
private solutions for the problem.
> You *can* write a successful OS where virtual and physical are related
> in a more complex way; and you can cope with driver-visible caches.
>
> In fact, the MIPS approach has a lot to recommend it. As well as
> making the hardware simpler, it's often more efficient for the CPU to
> writeback or invalidate the cache - the hardware poking from the other
> side ends up shutting the CPU out of the cache while it does the job.
>
> It's an interesting question how the "mainstream" Linux camp would
> react to the idea that this is a portable coding practice question,
> which is everybody's problem. Many useful memory systems are a lot
> more complicated than a PC-clone forget-the-cache approach can
> support; it's not just MIPS.
>
> Random points:
>
> I can't see how anything you do at kmalloc() time is going to fix the
> problem.
Linux's kmalloc has a second flag argument where one can pass flags like
GFP_DMA for DMA-able memory. As a simple fix kmalloc & Co. could then
return cacheline aligned memory with a KSEG1 address. Simple fix for
alot of drivers.
In case of the Wreckstation Tyne we really need to tweak the memory
allocation scheme - DMA-able memory comes from a different pool than
the normal memory. Nice side effect - you never have DMA cache coherency
problems on that machine.
> I decided a couple of manuals back that I'd give up using the term
> "cache flush" because I was never sure whether I (or anyone else)
> meant "invalidate" or "writeback" or both.
I agree. In the Linux kernel it does not matter in many cases whether
one does write back the cache or just invalidate it, so the used term
flush is ok, I think. Same for flush_page_to_ram, I think.
> > ... it might also make sense to have a portable way to flush caches.
> > For that I suggest an hook similar to the other cache flushing
> > functions.
>
> You really need two functions:
>
> writeback (addr, n)
>
> which on a vanilla MIPS machine would write back n bytes worth of
> locations from addr, and
>
> invalidate (addr, n)
>
See include/asm-mips/r4kcache.h:
extern inline void flush_icache_line_indexed(unsigned long addr);
extern inline void flush_dcache_line_indexed(unsigned long addr);
extern inline void flush_scache_line_indexed(unsigned long addr);
extern inline void flush_icache_line(unsigned long addr);
extern inline void flush_dcache_line(unsigned long addr);
extern inline void flush_scache_line(unsigned long addr);
> these are physical or virtual addresses doesn't matter much, so long
> as you decide which and don't change your mind! My preference would
> be to work in the kernel's normal address space for as long as
> possible, and provide a function for drivers wanting to calculate
> physical addresses...
>
> However, you wouldn't actually usually write those in drivers; you'd
> code them as:
>
> before_dma_out (addr, n) == writeback on MIPS
> after_dma_in (addr, n) == invalidate on MIPS
That's essentially my suggestion in my last posting about extending the
already available cache flushing functionality:
(from include/asm-mips/pgtable.h)
/* Function pointers - all the MIPS hardware is sooo similar ... */
extern void (*flush_cache_all)(void);
extern void (*flush_cache_mm)(struct mm_struct *mm);
extern void (*flush_cache_range)(struct mm_struct *mm, unsigned long start,
unsigned long end);
extern void (*flush_cache_page)(struct vm_area_struct *vma, unsigned long page);
extern void (*flush_cache_sigtramp)(unsigned long addr);
and then new:
extern void (*flush_cache_before_dma_out)(unsigned long start,
unsigned long length);
extern void (*flush_cache_after_dma_in)(unsigned long start,
unsigned long length);
The floppy driver already uses something similar, just abstracted out
another way into the header files.
> On a PC clone, both "before..." and "after..." can be ifdef'd away.
Ralf