Fog Creek Software
Discussion Board




FTP Relay?


I've got a 3-tier system on a standard dual firewall network:

Internet  |FW|  Web Server  |FW|  DB & App Servers

The application server works with files transferred in and out from other systems on the internet (similar to biztalk)

So the question - how should the files be moved from the 'net to the app server? What's the best practice in this situation? The most obvious solution for me is to have an FTP Relay system running on the web server, but googling "ftp relay" produces an amazing lack of hits, which makes me think not many people are doing this.

My suspicion is that most people just open Port 21 all the way through the DMZ, but that defeats the purpose of having a DMZ, doesn't it?

What say ye?

Philo

Philip Janus
Friday, February 21, 2003

What would your FTP relay do?

Just me (Sir to you)
Friday, February 21, 2003

Service running on the web server as a bidirectional relay. It has two FTP daemons running (on different ports) - one facing the 'net, one facing the internal network. When a file is FTP'd in to the external interface, then an FTP client connects to an FTP server on the application server and forwards the file.

Ditto on the outbound side.

I can write one - it's trivial; I'm just surprised that nobody else has.

Philo

Philip Janus
Friday, February 21, 2003

>What say ye?

I would open port 21 on the web-server and use scp or something getting the files from the web server into the application server. Thus disallow ftp traffic in the second line of firewall defence.

scp gives you remote copy over an ssl secured line, so you would need SSL handshake keys on the web server. But that is still better than plain ftp straight in.

Your post does not tell if you need real-time-relaying or if you can run a transfer batch of the files from the web server, but if scheduled batches will do its a no brainer to implement.

Another way would be setting up an email address on the web server for each person needing to send files, and then use procmail or similar to save the attached files and relay them to the application server via scp. This would give you real time relaying since procmail is triggered when incoming mail arrives. This is probably better since you can do away with opening the ftp-port all together.

Patrik
Friday, February 21, 2003

Still, it seems to me that accepting the file and indiscriminatly forwarding it does not give you much added security. Sure I understand you do not expose the ftpd on the app server to *.*.*.*, but if you can not trust the machine in the DMZ because you open up the ftpd there, what does it buy you?

Just me (Sir to you)
Friday, February 21, 2003

The files aren't the problem (tho granted, I had already thought about only forwarding ASCII and limiting file size) - the issue is FTP vulnerabilities. The Webserver is expendable, machines with data on them aren't...

At least, that's my thinking on it. Doesn't mean it's right. ;-)

Philo

Philip Janus
Friday, February 21, 2003

I think it depends on what you really want to do (an easy answer, I admit).

I'll try out two cases considering file transfert for your back-office :

1) Code or content upgrade done by few people mainly in the same company.

I'll recommand a VPN, preferably handled by your firewall.

2) Anonymous, standardised, file exchange for many sources.

I'll use a machine in the DMZ as a file buffer and then use a batch or program running in my trusted zone to get the file.

This allows you to take advantage from the stateful features of your FW (deny DMZ to Trusted access and grant Trusted to DMZ).

Hope it helps

Ralph Chaléon
Friday, February 21, 2003

As an answer to Just Me:
http://www.saintcorporation.com/demo/saint_tutorials/FTP_vulnerabilities.html

That's a list of some FTP vulnerabilities for FTPD, I'm sure similar exist for IIS. Some of them allow execution of random code or taking control as root.

If that happens on the webserver, then you've lost one machine with no data (if you're doing it right). If it happens on your app server, which might have access to your database server and your internal network, you're fuct.

Philo

Philip Janus
Friday, February 21, 2003

How aboot having the App server grab the files from the Web server on a scheduled basis?

Neil E
Friday, February 21, 2003

Philo,

I agree whith this "ftpd is a risk" reasoning, and that is exactly why I said your setup with the FTP relay does not seem to buy you much.
If your Webserver gets rooted because of an ftpd exploit, the attacker then has control over the machine can use it to exploit the vulnerabilities in your ftpd on the appserver.

Contrast this with a setup along the lines of what Ralph described where a process from inside reaches out into the DMZ, and you eliminate the domino effect.

Just me (Sir to you)
Friday, February 21, 2003

Philo, surely if an attacker can gain control of your webserver thru ftpd (or whatever) then you're lost anyway if you are running that same software behind the second FW? He would gain control of your webserver, install whatever he wants there, and then start scanning whatever it is connected to. Of course, network security is not my business so I may be missing something as well! Surely (and yes, I know, I hate polling too) it would be better to have the system behind the firewall connect to the ftp/webserver and download the files?

A Chocolate Orange
Friday, February 21, 2003

Ach, Ignore me. I missed or misread your description of what your ftp relay would do!

Still A Chocolate Orange
Friday, February 21, 2003

The problem with scp, is that it would negate the benefits of having a DMZ.  You'd need to set up some passwordless keys so that the program could scp stuff into the interior db server.  But, then whenever someone broken into the web server, they could just ssh straight into the db server w/out a password.  voila!  through the DMZ with little to no work.  Don't use scp.  Or if you do, make sure that you use an intern to scp the files by hand regularly ;)

hmm... even if you ftp, your program still needs to know the password.  And then anyone on your webserver gets past the dmz.  Why isn't your database server in the dmz, just firewalled off from everything but the web server?

Andrew Hurst
Friday, February 21, 2003

After re-reading the above responses, I'd like to second (or fifth) that opinion that the db server should just pull the files from the web server.  Sounds like the best solution to me.

Andrew Hurst
Friday, February 21, 2003

I second Ralph's idea.

In case you have access to the DB server in
the internet network via DB access, you
might write a thin web application that
specializes in web uploads. That way the web
server in the DMZ write the blob of binary
stream to the DB server in the internal
network. It will have all the SSL goodness
and password protection.. just get it
audited and peer reviewed before going live.
You can even turn it into a web service if
XMLRPC/SOAP will support it.

The tabbed interface should be:

vvvvvvvvvvv
/ Please Log-in \ / Upload \ / Download \/ Log off \ (UL/DL/Log off grayed out)

Enter username: _________
Enter password: _________


To upload:
              vvvvvvvv
/ Logged In \ / Upload \ / Download \/ Log off \

Pick a file:

[Click to upload] <- button to start upload

To show upload success:

        Say it's done, and send an email
notification to someone on the internal network.

To complain:

        Say you are reusing filenames, or
something, , and show the pick file screen again.

To download:

              vvvvvvvv
/ Logged In \ / Upload \ / Download \/ Log off \

Show a listing of files available.

[Click to download selected file] <-= button to start download

To show download success:

        Say it's in progress... <-- then send an
email notification to person who uploaded
the file and the file transfer admins.. that
another person downloaded it.
In the back end you need the following table
and fields:

        Users (UserID, GroupID, username, password, emailaddress)

        Files (FileID, OwnerID/GroupID, Blob, Size)

        (optionally you can add groups and visibility scoping)

        GroupID (GroupID, description)

Go grab a file upload module (comes with
Perl and PHP already) for asp and make it a
weekend project. Generate a private SSL
server/client key certification and have
fun! :-)

Adding an administration interface will make
adding/deleting users and files easier. A
few day's work at the very worst.


But yeah.. some people don't describe this
solution using the keyword "ftp relay".. so
flex your search a bit. Try "ftp dmz" "ftp
upload dmz" or something. What do you think?

Li-fan Chen
Friday, February 21, 2003

"If your Webserver gets rooted because of an ftpd exploit, the attacker then has control over the machine can use it to exploit the vulnerabilities in your ftpd on the appserver."

He wouldn't be running ftpd on the appserver, would he?  Even if he chooses to use the ftp relaying solution, won't it just be an ftp client on the appserver?  The only ftp server that's running is on the webserver.  So that's the only one that could get hacked through ftpd.  Or am I missing something?

Herbert Sitz
Friday, February 21, 2003

Herbert,

as it was explained by the original poster:

"When a file is FTP'd in to the external interface, then an FTP client connects to an FTP server on the application server and forwards the file"

that was the actual problem with the setup.

Just me (Sir to you)
Friday, February 21, 2003

Sir -- Gotcha.  OK, then I agree with you completely.  If they hacked into webserver via ftp, then what would stop them from doing the same hack to get onto the app server?

Herbert Sitz
Friday, February 21, 2003

Just kinda thinking out loud here (I reserve the right to be wrong at any time)......

1.  File uploaded to web/dmz server.
2.  When new file received, web server sends MSMQ message to DB/App Server over secured port to a Private Message Queue on DB/App server.
3.  DB/App server receives MSMQ message and reaches out to Web/DMZ server and grabs file.
4.  Everyone celebrates at success of new system.
5.  Philo gets big raise.

...or maybe not.  Thoughts?

Jeff MacDonald
Friday, February 21, 2003

There is a python script that comes with the ActivePython distribution that does FTP mirroring for you.

Or maybe look into rsync if you either are running *nix or Cygwin.

Damian
Friday, February 21, 2003

Excuse my total ignorance, but what is MSMQ?

I want a raise too :)
Saturday, February 22, 2003

www.google.com

The solution to all your ignorance needs ;)

Damian
Saturday, February 22, 2003

MSMQ: Microsoft Message Queue.

Think of it as email for computer systems - a way for systems (or components) to send messages to one another and organize/act upon those messages.

http://www.microsoft.com/msmq/

Jeff MacDonald
Saturday, February 22, 2003

>MSMQ: Microsoft Message Queue.

Oh, OK. I was confused about the "MS" part. I was thinking MQ was IBM MQ-Series which is also a message queueing product.

Thanks a bunch.

I want a raise too
Saturday, February 22, 2003

...

I'm trying to come to grips with a member of this board not knowing what "MS" stands for...

[grin]

Philo

Philip Janus
Saturday, February 22, 2003

>I'm trying to come to grips with a member of this board
>not knowing what "MS" stands for...

Hahaha...you got me. In all fairness, I have to learn how to not walk into these traps. Im going to apply to "Make yourself understood class 1A". Seems needed ;-)


[grin]

Patrik
Saturday, February 22, 2003

Philo - I'm curious, what did you eventually do?


Friday, February 28, 2003

*  Recent Topics

*  Fog Creek Home